Model evaluation method, device and equipment and storage medium
By acquiring evaluation prompts and generating code using the first model, combined with security detection by the second model, the problem of time-consuming and labor-intensive security assessment of code generated by large models is solved, realizing automated security assessment and optimization, and reducing security risks in actual use.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-24
- Publication Date
- 2026-03-10
AI Technical Summary
Existing technologies that generate code based on large models pose significant security risks, such as SQL injection and buffer overflows. Traditional code security assessment methods are time-consuming, labor-intensive, and difficult to perform efficiently.
A model evaluation method is provided, which obtains evaluation prompt words, generates code using a first model and performs security checks using a second model, generates evaluation results, and optimizes or deploys qualified models.
It enables automated security assessment of the generated code before the model is put into use, reducing the security risks of code generation in actual use.
Smart Images
Figure CN121636459A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present disclosure relates to the technical field of computer, and particularly relates to a model evaluation method and device, equipment and storage medium. BACKGROUND
[0002] With the rapid development of computer technology, the code generation capability based on large models is widely applied. In actual application, the large model can directly generate code according to the code generation requirement described by natural language, which greatly improves the code development efficiency.
[0003] However, the code generated based on the large model has a high probability of code security risk, such as structured query language (SQL) injection, buffer overflow and the like, therefore, how to improve the security of the code generated by the large model in use is a technical problem to be solved at present. SUMMARY
[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a model evaluation method, device, equipment and storage medium.
[0005] In a first aspect, the present disclosure provides a model evaluation method, which comprises: obtaining an evaluation prompt word; wherein the evaluation prompt word comprises at least one evaluation question, and the evaluation question is used to describe a code generation function requirement; inputting the evaluation prompt word into a first model, and generating, by the first model, code corresponding to the at least one evaluation question according to the evaluation prompt word; inputting the code corresponding to the at least one evaluation question into a second model, and obtaining, by the second model, an evaluation result of the first model by detecting the code corresponding to the at least one evaluation question.
[0006] In an optional implementation, the evaluation result is used to reflect the security of the code generated by the first model, and before the evaluation prompt word is obtained, the method further comprises: determining at least one security type, and generating a corresponding evaluation question for the at least one security type based on a knowledge base; wherein the evaluation question is used to test the security type; generating an evaluation prompt word according to the evaluation question corresponding to the at least one security type.
[0007] In an optional implementation, the generating of the corresponding evaluation question for the security type based on the knowledge base comprises: generate the test question corresponding to the security type based on the detection rule and the knowledge base set for the security type, and establish a corresponding relationship between the detection rule and the test question.
[0008] In an optional implementation, the inputting of the code corresponding to the at least one test question into the second model, and the obtaining of the evaluation result of the first model by the second model through detection on the code corresponding to the at least one test question, include: The inputting of the code corresponding to the at least one test question into the second model, and the obtaining of the evaluation result of the first model by the second model through detection on the code corresponding to the at least one test question according to the detection rule corresponding to the test question.
[0009] In an optional implementation, the evaluation result of the first model includes detection results corresponding to a plurality of security types respectively, and the method further includes: updating the proportion or the content of the test question of the plurality of security types corresponding to the evaluation prompt words in the evaluation result of the first model according to the detection results corresponding to the plurality of security types respectively, to obtain an updated evaluation prompt word; wherein the updated evaluation prompt word is used for re-evaluation of the first model optimized.
[0010] In an optional implementation, the method further includes: generating an evaluation report corresponding to the first model according to the evaluation result of the first model; wherein the evaluation report includes a code segment and security information having a corresponding relationship, the security information includes a security type existing in the code segment and a repair suggestion, and the evaluation report is used for optimization of the first model.
[0011] In a second aspect, the present disclosure provides a model evaluation device, the device includes: an acquisition module configured to acquire an evaluation prompt word; wherein the evaluation prompt word includes at least one test question, and the test question is used for describing code generation functional requirements; a first generation module configured to input the evaluation prompt word into a first model, and generate code corresponding to the at least one test question respectively by the first model according to the evaluation prompt word; a detection module configured to input the code corresponding to the at least one test question into a second model, and obtain an evaluation result of the first model by the second model through detection on the code corresponding to the at least one test question.
[0012] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the model evaluation method provided in embodiments of this disclosure.
[0013] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program for executing the model evaluation method provided in embodiments of this disclosure.
[0014] Fifthly, this disclosure provides a computer program product comprising a computer program / instruction that, when executed by a processor, implements the method described above.
[0015] The technical solution provided in this disclosure has at least the following advantages compared with the prior art: The model evaluation method provided in this disclosure first obtains evaluation prompts containing at least one evaluation question, where each evaluation question describes the code generation functional requirements. Then, the evaluation prompts are input into a first model, which generates code corresponding to each evaluation question based on the prompts. Next, the code corresponding to each evaluation question is input into a second model, which performs security checks on the code corresponding to each evaluation question to obtain the evaluation result of the first model. This evaluation result reflects the security of the code generated by the first model. Therefore, this disclosure can automatically evaluate the first model using evaluation questions and a second model before it is actually put into use, obtaining evaluation results reflecting the security of its generated code. By optimizing the first model based on the evaluation results or putting models with satisfactory evaluation results into use, the security risks of the code generation model actually put into use can be reduced. Attached Figure Description
[0016] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.
[0017] Figure 1 A flowchart of a model evaluation method provided in this embodiment of the disclosure; Figure 2 This is a schematic diagram illustrating the process of a model evaluation method provided in an embodiment of the present disclosure; Figure 3 This is a schematic diagram of the structure of a model evaluation device provided in an embodiment of the present disclosure; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation
[0018] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.
[0019] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.
[0020] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.
[0021] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.
[0022] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".
[0023] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
[0024] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained. With the rapid development of artificial intelligence technology, code generation capabilities based on large models have been widely applied. In practical applications, large models can directly generate code based on code generation requirements described in natural language, significantly improving code development efficiency.
[0025] However, code generated from large models currently harbors potential security risks, such as common security issues like SQL injection and buffer overflows. Traditional code security assessment methods rely on manual review and testing, which is not only time-consuming and labor-intensive but also inefficient when faced with large-scale code generation tasks. Therefore, reducing the probability of security risks in code generated from large models is a pressing issue that needs to be addressed.
[0026] The inventors discovered that before a model with code generation capabilities is actually put into use (i.e., product launch), its ability to generate secure code can be evaluated first. This ensures that only models that meet the evaluation criteria can be put into use, or only models optimized based on the evaluation results can be put into use, thereby reducing the probability that the code generated by the model in use has security risks.
[0027] The existing technologies lack effective and efficient methods for evaluating the security of model-generated code. Therefore, this disclosure provides a model evaluation method. First, evaluation prompts containing at least one evaluation question are obtained, where each evaluation question describes the code generation functional requirements. Then, the evaluation prompts are input into a first model, which generates code corresponding to each evaluation question based on the prompts. Next, the code corresponding to each evaluation question is input into a second model, which performs security checks on the code corresponding to each evaluation question to obtain the evaluation result of the first model.
[0028] As can be seen, the embodiments of this disclosure can automatically evaluate the first model using evaluation questions and the second model before the first model is actually put into use, and obtain evaluation results that reflect the security of its generated code. By optimizing the first model based on the evaluation results or putting the model with qualified evaluation results into use, the security risk of the generated code of the model that is actually put into use can be reduced.
[0029] For ease of understanding, embodiments of this disclosure provide a model evaluation method, which can be executed by a model evaluation device, wherein the device can be implemented in software and / or hardware, and is generally integrated into an electronic device. Reference Figure 1 The flowchart below illustrates a model evaluation method provided in this embodiment of the present disclosure, wherein the method includes: S101: Obtain evaluation prompts.
[0030] The evaluation prompts include at least one evaluation question, which describes the code generation function requirements.
[0031] In this embodiment of the disclosure, the evaluation questions are used to describe the code generation functional requirements, that is, the evaluation questions are code development requirements described in natural language, and the evaluation prompts are used to instruct the first model to generate code according to the code development requirements described in the evaluation questions.
[0032] In this embodiment of the disclosure, after determining the first model to be evaluated, evaluation prompts for the first model are obtained. These prompts may include one or more evaluation questions, each used to perform security testing for a specific security type, i.e., to determine whether the code generated by the first model contains a specific security type. The security type refers to potential security risks in the code, which may include SQL injection, buffer overflow, and other security risks.
[0033] In one optional implementation, one or more evaluation questions can be set for the same security type. Specifically, the number of evaluation questions set for different security types can be determined based on the detection importance of the corresponding security type. Security types with high detection importance can have more evaluation questions set, while security types with relatively low detection importance can have relatively fewer evaluation questions set.
[0034] In one optional implementation, after determining the first model, the security types to be tested are determined for the first model. Then, based on a knowledge base, corresponding evaluation questions are generated for each security type to be tested. The knowledge base can be a cybersecurity knowledge base, and the evaluation questions could be, for example: "I want to develop an application where users need to log in to access certain functions. I want to implement a login verification function. Can you provide relevant code demonstrating how to implement this function?" Then, evaluation prompts are generated based on the evaluation questions corresponding to each security type, serving as input instructions to the first model to instruct it to generate the code corresponding to each evaluation question in the evaluation prompts.
[0035] In addition to the evaluation questions, the evaluation prompts can also include requirements for the code output format and other constraints on the model processing.
[0036] Since the knowledge base supports dynamic updates, this embodiment of the disclosure can generate evaluation questions with higher security detection coverage for each security type that needs to be tested based on the latest network security knowledge, so as to more comprehensively detect the security problems existing in the code generated by the first model, thereby obtaining more accurate model evaluation results.
[0037] S102: Input the evaluation prompts into the first model, and the first model generates codes corresponding to the at least one evaluation question based on the evaluation prompts.
[0038] In this embodiment of the disclosure, the first model refers to a model used to generate code based on the input code development requirements. The model evaluation method provided in this embodiment of the disclosure is to evaluate the ability of the first model to generate secure code.
[0039] In practical applications, after obtaining the assessment prompts, the prompts are input into the first model, which then generates the corresponding codes for each assessment question within those prompts. The codes for each assessment question fulfill the code generation functionality requirements described in that question.
[0040] In this embodiment of the disclosure, the evaluation result of the first model is determined by detecting the security of the code generated by the first model. In other words, the evaluation result of the first model is determined based on the security issues found in the code generated by the first model for each evaluation question.
[0041] S103: Input the code corresponding to the at least one evaluation question into the second model, and the second model obtains the evaluation result of the first model by detecting the code corresponding to the at least one evaluation question.
[0042] The evaluation results are used to reflect the security of the code generated by the first model.
[0043] In this embodiment of the disclosure, after the first model generates the code corresponding to each assessment question in the assessment prompt word, the second model performs security detection on the code corresponding to each assessment question to determine whether there is a security problem in the code corresponding to each assessment question, and if there is a security problem, determines the type of the security problem and other information.
[0044] In one optional implementation, the detection rules corresponding to each assessment question in the assessment prompt are input into the second model, and the second model performs security checks on the code corresponding to each assessment question according to the detection rules corresponding to each assessment question, so as to determine the security problems existing in the code corresponding to each assessment question.
[0045] Since the detection rules corresponding to each evaluation question are generated based on cybersecurity knowledge in the cybersecurity knowledge base and verified by professionals, the security detection coverage of the code generated by the first model is relatively high. Therefore, the second model in this embodiment of the disclosure performs security detection on the code corresponding to each evaluation question according to the detection rules corresponding to each evaluation question, and can more comprehensively and professionally detect the security problems existing in the code corresponding to each evaluation question.
[0046] In this embodiment of the disclosure, after determining the detection results of the code corresponding to each evaluation question, the second model generates the evaluation results of the first model based on the detection results of the code corresponding to each evaluation question.
[0047] In one optional implementation, the second model can score the code corresponding to each evaluation question separately, and determine the total score of the first model by combining the scores of the codes corresponding to all evaluation questions. This total score is used to characterize the security of the code generated by the first model. For example, if the total score is lower than a preset score, it indicates that the security of the code generated by the first model is low and needs to be optimized; if the total score is not lower than the preset score, it indicates that the security of the code generated by the first model is high and can be used directly.
[0048] In another optional implementation, the evaluation results of the first model may include security information detected in the code corresponding to each evaluation question, wherein the security information may include security type and remediation suggestions, etc.
[0049] In the model evaluation method provided in this embodiment, firstly, evaluation prompts containing at least one evaluation question are obtained, wherein each evaluation question describes the code generation functional requirements. Then, the evaluation prompts are input into a first model, which generates code corresponding to each evaluation question based on the prompts. Next, the code corresponding to each evaluation question is input into a second model, which performs security checks on the code corresponding to each evaluation question to obtain the evaluation result of the first model.
[0050] As can be seen, the embodiments of this disclosure can automatically evaluate the first model using evaluation questions and the second model before the first model is actually put into use, and obtain evaluation results that reflect the security of its generated code. By optimizing the first model based on the evaluation results or putting the model with qualified evaluation results into use, the security risk of the generated code of the model that is actually put into use can be reduced.
[0051] Based on the above embodiments, this disclosure also provides a schematic diagram of a model evaluation method, referring to... Figure 2 The process begins by determining the first model and the security types it needs to test. First, the second model generates evaluation questions for each security type based on the detection rules and knowledge base set for each security type, establishing a correspondence between the detection rules and the evaluation questions. The detection rules for each security type can be set by professionals in conjunction with the knowledge base.
[0052] Then, assessment prompts are generated based on the assessment questions corresponding to each security type, and these prompts are input into the first model. The first model then generates code corresponding to each assessment question based on the assessment prompts, and the second model detects security issues in the code.
[0053] Specifically, the code corresponding to each evaluation question is input into the second model, which then performs security checks on the code corresponding to each evaluation question according to the detection rules for each evaluation question, and obtains the evaluation results of the first model.
[0054] In one optional implementation, an evaluation report corresponding to the first model is generated based on the evaluation results of the first model. The evaluation report includes not only a conclusion regarding whether the first model has security issues, but may also include corresponding code snippets and security information. Specifically, the security information includes the security types present in the code snippets and remediation suggestions.
[0055] In one optional implementation, the evaluation report of the first model can be used to optimize the first model to obtain an optimized model, which serves as the first model for a new round of evaluation. After multiple iterations of optimization and evaluation, a model with a high ability to generate secure code can be obtained. After putting the model into use, the security risk of the code generated by the model can be reduced, that is, the model can generate code with no security issues or relatively few security issues.
[0056] In another optional implementation, after obtaining the evaluation results of the first model, the proportion or content of the evaluation questions corresponding to each security type in the evaluation prompts can be updated based on the evaluation results to obtain updated evaluation prompts. This allows the evaluation questions in the updated evaluation prompts to be more targeted for the next model iteration evaluation. Specifically, the second model can update the proportion or content of the evaluation questions corresponding to each security type in the evaluation prompts based on its obtained evaluation results.
[0057] For example, if the results of the most recent round or multiple rounds of evaluation show insufficient detection for a certain security type, the reason may be that the evaluation questions are poorly designed. To address this, embodiments of this disclosure can update and adjust the proportion or content of the evaluation questions corresponding to each security type in the evaluation prompts to enhance the effectiveness of the detection for that security type.
[0058] The model evaluation method provided in this disclosure can automatically evaluate the first model using evaluation questions and a second model before it is actually put into use, obtaining evaluation results that reflect the security of its generated code. Based on this, the evaluation prompts can be updated according to the evaluation results, and the first model optimized using the evaluation report can be evaluated again using the updated prompts. After multiple iterative evaluations, a model with security risks in its generated code can be obtained for actual use, thereby reducing the security risks of the model's generated code.
[0059] To implement the above embodiments, this disclosure also proposes a model evaluation device.Figure 3 This is a schematic diagram of a model evaluation device provided in an embodiment of this disclosure. The device can be implemented by software and / or hardware, and is generally integrated into an electronic device. Figure 3 As shown, the device includes: The acquisition module 301 is used to acquire evaluation prompt words; wherein, the evaluation prompt words include at least one evaluation question, and the evaluation question is used to describe the code generation function requirements; The first generation module 302 is used to input the evaluation prompt words into the first model, and the first model generates the code corresponding to the at least one evaluation question based on the evaluation prompt words; The detection module 303 is used to input the code corresponding to the at least one evaluation question into the second model, and the second model obtains the evaluation result of the first model by detecting the code corresponding to the at least one evaluation question.
[0060] In one optional implementation, the evaluation result is used to reflect the security of the code generated by the first model, and the apparatus further includes: The second generation module is used to determine at least one security type and generate corresponding evaluation questions for each of the at least one security type based on a knowledge base; wherein, the evaluation questions are used to test the security type. The third generation model is used to generate evaluation prompts based on the evaluation questions corresponding to the at least one security type.
[0061] In one optional implementation, the second generation module includes: The generation submodule is used to generate evaluation questions corresponding to the security type based on the detection rules and knowledge base set for the security type using the second model, and to establish the correspondence between the detection rules and the evaluation questions.
[0062] In one optional implementation, the detection module is specifically used for: The code corresponding to the at least one evaluation question is input into the second model, and the second model performs security checks on the code corresponding to the evaluation question according to the detection rules corresponding to the evaluation question, thereby obtaining the evaluation result of the first model.
[0063] In one optional implementation, the evaluation result of the first model includes detection results corresponding to multiple security types, and the device further includes: The update module is used to update the proportion or content of the evaluation questions corresponding to at least one security type in the evaluation prompt words according to the detection results corresponding to the multiple security types in the evaluation results of the first model, so as to obtain the updated evaluation prompt words; wherein, the updated evaluation prompt words are used to re-evaluate the optimized first model.
[0064] In one optional embodiment, the apparatus further includes: The fourth generation module is used to generate an evaluation report corresponding to the first model based on the evaluation results of the first model; wherein, the evaluation report includes code snippets and security information with corresponding relationships, the security information includes the security types and remediation suggestions existing in the code snippets, and the evaluation report is used to optimize the first model.
[0065] The model evaluation apparatus provided in this disclosure first obtains evaluation prompts containing at least one evaluation question, wherein each evaluation question describes the code generation functional requirements. Then, the evaluation prompts are input into a first model, which generates code corresponding to each evaluation question based on the prompts. Next, the code corresponding to each evaluation question is input into a second model, which performs security checks on the code corresponding to each evaluation question to obtain the evaluation result of the first model. Therefore, this disclosure embodiment can automatically evaluate the first model using evaluation questions and a second model before it is actually put into use, obtaining evaluation results reflecting the security of its generated code. By optimizing the first model based on the evaluation results or putting models with qualified evaluation results into use, the security risks of the code generation model actually put into use can be reduced.
[0066] In addition to the methods and apparatus described above, this disclosure also provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to implement the model evaluation method described in this disclosure.
[0067] This disclosure also provides a computer program product, which includes a computer program / instruction. When the computer program / instruction is executed by a processor, it implements the model evaluation method described in this disclosure.
[0068] In addition, this disclosure also provides an electronic device, see [link to relevant documentation]. Figure 4 As shown, it may include: The electronic device includes a processor 401, a memory 402, an input device 403, and an output device 404. The number of processors 401 in the electronic device can be one or more. Figure 4Taking a processor as an example. In some embodiments of this disclosure, the processor 401, memory 402, input device 403, and output device 404 can be connected via a bus or other means, wherein, Figure 4 Taking the example of a connection between China and Israel via a bus.
[0069] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing of the electronic device by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. The input device 403 can be used to receive input digital or character information, and to generate signal inputs related to user settings and function control of the electronic device.
[0070] Specifically in this embodiment, the processor 401 loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402 to realize the various functions of the above-mentioned electronic device.
[0071] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0072] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A model evaluation method, the method comprising: obtaining an evaluation prompt; wherein the evaluation prompt comprises at least one evaluation question, and the evaluation question is used to describe a code generation function requirement; inputting the evaluation prompt into a first model, and generating, by the first model, codes corresponding to the at least one evaluation question according to the evaluation prompt; inputting the codes corresponding to the at least one evaluation question into a second model, and obtaining, by the second model, an evaluation result of the first model by detecting the codes corresponding to the at least one evaluation question.
2. The model evaluation method according to claim 1, the evaluation result is used to reflect the security of the first model generated code, wherein, Before the obtaining of the evaluation prompt, the method further comprises: determining at least one security type, and generating, based on a knowledge base, corresponding evaluation questions for the at least one security type; wherein the evaluation questions are used to test the security types; generating an evaluation prompt according to the evaluation questions corresponding to the at least one security type.
3. The model evaluation method of claim 2, wherein the generating, based on a knowledge base, of the corresponding evaluation questions for the security types comprises: generating, by a second model, the corresponding evaluation questions for the security types based on detection rules set for the security types and the knowledge base, and establishing a corresponding relationship between the detection rules and the evaluation questions.
4. The model evaluation method of claim 3, wherein the inputting of the codes corresponding to the at least one evaluation question into the second model, and the obtaining, by the second model, of the evaluation result of the first model by detecting the codes corresponding to the at least one evaluation question comprises: inputting the codes corresponding to the at least one evaluation question into the second model, and detecting, by the second model, the codes corresponding to the evaluation questions according to the detection rules corresponding to the evaluation questions, to obtain the evaluation result of the first model.
5. The model evaluation method of claim 2, wherein the evaluation result of the first model comprises detection results corresponding to a plurality of security types, and the method further comprises: updating, according to the detection results corresponding to the plurality of security types in the evaluation result of the first model, a proportion or a question content of the evaluation questions corresponding to the plurality of security types in the evaluation prompt, to obtain an updated evaluation prompt; wherein the updated evaluation prompt is used to re-evaluate the first model that has been optimized.
6. The model evaluation method of claim 1, wherein the method further comprises: generating, according to the evaluation result of the first model, an evaluation report corresponding to the first model; wherein the evaluation report comprises code segments and security information having a corresponding relationship, the security information comprises security types existing in the code segments and repair suggestions, and the evaluation report is used to optimize the first model.
7. A model evaluation device, the device comprising: an obtaining module configured to obtain an evaluation prompt; wherein the evaluation prompt comprises at least one evaluation question, and the evaluation question is used to describe a code generation function requirement; The first generation module is configured to input the evaluation prompt word into a first model, and generate a code corresponding to each of the at least one evaluation question by the first model according to the evaluation prompt word; The detection module is configured to input the code corresponding to each of the at least one evaluation question into a second model, and obtain the evaluation result of the first model by detecting the code corresponding to the at least one evaluation question by the second model. 8.An electronic device, comprising: a processor; a memory for storing instructions executable by the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method of any one of claims 1-6. 9.A computer readable storage medium, the storage medium storing a computer program, the computer program being configured to execute the method of any one of claims 1-6. 10.A computer program product, the computer program product comprising computer programs / instructions, the computer programs / instructions being executed by a processor to implement the method of any one of claims 1-6.