Target language model security evaluation method and electronic device

Abstract

The application provides a target language model security evaluation method and electronic equipment, which can be applied to the network space security technical field. The method comprises the following steps: based on a security level classification standard, performing security level classification on multiple security fields to obtain a classification result; for each security level of the multiple security levels, performing test question bank construction on at least one security field to obtain a test question bank; performing model application security testing on a target language model according to at least one test question in an attack test question bank and at least one test question in a refusal test question bank to obtain a model application security testing result; performing model function security testing on the target language model based on a risk capability test case to obtain a model function security testing result, wherein the risk capability test case is used for testing the function security risk of the target language model; and generating a security evaluation report of the target language model according to the model application security testing result and the model function security testing result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cyberspace security, in particular to a target language model security evaluation method and an electronic device. BACKGROUND

[0002] In recent years, the development of large language models has attracted more and more attention, and has shown great application potential in education, medical treatment, finance and other fields. However, with the wide deployment and application of large models, the security of their output content is attracting more and more attention. Large models are usually trained on a large amount of text data, and due to the lack of proper supervision of these data, it is difficult for large models to avoid generating content that violates values, bringing potential misuse risks. Therefore, it is important to comprehensively and synthetically evaluate the security of large language models.

[0003] However, the existing security benchmarks involve limited security risk scenarios, and lack unified and comprehensive large model security risk evaluation indicators. Secondly, the simple question-asking method limits the reflection of the security level of large models in different security scenarios, and the accuracy of evaluating large language models is low, and the model misuse risk is high. SUMMARY

[0004] In view of the above problems, the present application provides a target language model security evaluation method and an electronic device.

[0005] According to a first aspect of the present application, a target language model security evaluation method is provided, comprising: classifying a plurality of security fields according to a security level classification standard to obtain a classification result, wherein the classification result includes at least one security field corresponding to each of the plurality of security levels; for each security level of the plurality of security levels, constructing a test question bank for at least one security field to obtain a test question bank, wherein the test question bank includes at least an attack test question bank and a refusal test question bank, the attack test question bank is used to test the security defense capability of the target language model when facing sensitive word camouflage attack, and the refusal test question bank is used to test the refusal ability of the target language model when facing risk content questioning; performing model application security testing on the target language model according to at least one test question in the attack test question bank and at least one test question in the refusal test question bank to obtain a model application security testing result; performing model function safety testing on the target language model based on a risk ability test case to obtain a model function safety testing result, wherein the risk ability test case is used to test the function safety risk of the target language model; and generating a security evaluation report of the target language model according to the model application security testing result and the model function safety testing result.

[0006] Optionally, the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, including: inputting the attack test question bank into the target language model to output a first question and answer result; inputting the rejection test question bank into the target language model to output a second question and answer result; performing model application security evaluation analysis on the first question and answer result and the second question and answer result based on the evaluator to obtain the model application security test result.

[0007] Optionally, the model application security test result is obtained by performing model application security evaluation analysis on the first question and answer result and the second question and answer result based on the evaluator, including: obtaining a model attack success score according to the number of unidentified sensitive word camouflage attacks determined from the first question and answer result and the number of test questions in the attack test question bank; obtaining a model rejection ability score according to the number of rejections determined from the second question and answer result and the number of test questions in the rejection test question bank; and obtaining the model application security test result according to the model attack success score and the model rejection ability score of each of the plurality of security levels.

[0008] Optionally, the test question bank further includes a subjective test question bank and an objective test question bank; and the target language model security evaluation method further includes: inputting the subjective test question bank into the target language model to output a third question and answer result; inputting the objective test question bank into the target language model to output a fourth question and answer result; and performing model security evaluation analysis on the first question and answer result, the second question and answer result, the third question and answer result, and the fourth question and answer result based on the evaluator to obtain the model application security test result.

[0009] Optionally, the risk capability test case at least includes a data security test case, a model security test case, a system security test case, and a compliance environment test case; and the model function security test result is obtained by performing model function security test on the target language model based on the risk capability test case, including: performing security vulnerability test on data associated with the target language model according to a plurality of data security test subcases in the data security test case to obtain a data security score; performing security vulnerability test on the target language model according to a plurality of model security test subcases in the model security test case to obtain a model security score; performing security vulnerability test on a running system associated with the target language model according to a plurality of system security test subcases in the system security test case to obtain a system security score; performing security vulnerability test on the running compliance of the target language model according to a plurality of compliance environment test subcases in the compliance environment test case to obtain a compliance security score; and obtaining the model function security test result according to the data security score, the model security score, the system security score, and the compliance security score.

[0010] Optionally, according to the plurality of model security test subcases in the model security test case, the security vulnerability of the target language model is tested to obtain a model security score, including: according to the plurality of model security test subcases, the security vulnerability of the target language model is tested to obtain a test result corresponding to each model security test subcase; in a case where it is determined that at least one test result represents that the target language model has a security vulnerability, according to the number of affected system functions, the number of affected users and the number of affected data types caused by the at least one security vulnerability, a vulnerability impact range score and a vulnerability impact value score are obtained; according to the number of security vulnerabilities, the vulnerability impact range score and the vulnerability impact value score corresponding to each of the at least one security vulnerability, a model security score is obtained.

[0011] Optionally, the model security test subcase includes a model theft protection test subcase; wherein, according to the plurality of model security test subcases, the security vulnerability of the target language model is tested to obtain a test result corresponding to each model security test subcase, including: according to the question and answer result or the interface call result of the target language model, the model parameters of the target language model are reversely inferred to obtain the model parameters; a reference language model is generated based on the model parameters; in a case where the performance difference between the reference language model and the target language model is less than a preset threshold, a test result of a security vulnerability is obtained, wherein the test result of the security vulnerability represents that the model parameters of the target language model have a theft risk.

[0012] Optionally, the model security test subcase further includes at least one of the following: a model behavior monitoring test subcase, a model update test subcase, an adversarial attack test subcase, and a model virus attack defense test subcase.

[0013] Optionally, according to the model application security test result and the model function security test result, a security evaluation report of the target language model is generated, including: according to the model function security test result and a preset risk level mapping relationship, a model function security risk level is determined; according to the model function security risk level, the model application security test result and a vulnerability patching strategy corresponding to the security vulnerability of the target language model, a security evaluation report is generated, wherein the vulnerability patching strategy is generated for the test result representing that the target language model has a security vulnerability according to the data security test subcase, the model security test subcase, the system security test subcase and the compliance environment test subcase.

[0014] The second aspect of the application provides an electronic device, including: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned target language model security evaluation method.

[0015] According to the target language model security evaluation method and the electronic device provided by the application, the model application security test is performed on the target language model according to at least one test question in the attack test question bank and at least one test question in the refusal test question bank, and the model application security test result is obtained. The model function security test is performed on the target language model based on the risk capability test case, and the model function security test result is obtained. The safety evaluation report of the target language model is generated according to the model application security test result and the model function security test result. Since a safety evaluation framework of multiple safety fields and multiple scenes is constructed, the attack test question bank and the refusal test question bank are constructed according to the characteristics of the safety fields corresponding to different safety levels to perform the application safety evaluation on the model output content, and the safety protection capability of the target language model is accurately understood. The risk safety evaluation on the model function is performed by constructing multiple risk capability test case indexes, the checking capability of the safety vulnerability of the response content of the model and the system allowed to carry the model is accurately evaluated, and finally the comprehensive safety evaluation report is generated by comprehensively evaluating the model function security test result and the model application security test result, and the model misuse risk is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0016] The above and other objects, features and advantages of the present application will become more apparent from the following description of embodiments of the present application taken in conjunction with the accompanying drawings.

[0017] Figure 1 A flowchart of the target language model security evaluation method according to an embodiment of the present application is shown.

[0018] Figure 2 An example diagram of the risk capability test case according to an embodiment of the present application is shown.

[0019] Figure 3 An example diagram of the target language model security evaluation method according to an embodiment of the present application is shown.

[0020] Figure 4 A block diagram of the electronic device suitable for implementing the target language model security evaluation method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0021] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It should be understood, however, that the description which follows is merely exemplary and is not intended to limit the scope of the application. In the following detailed description of embodiments of the present application, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that one or more embodiments of the present application can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring the concepts of the present application.

[0022] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the terms "comprises", "comprising", "includes", "including" and the like are specifically intended to be open-ended terms meaning that other elements can be added.

[0023] All terms used herein including technical and scientific terms have the meanings commonly understood by one of ordinary skill in the art unless otherwise specified. It should be noted that the use of any terms herein should not be interpreted to limit the scope of the present disclosure to only those embodiments described herein. Rather, the terms should be interpreted broadly to include any embodiment that falls within the scope of the present disclosure.

[0024] In the case of using expressions similar to "at least one of A, B, and C, etc.", it is generally intended to include each and every combination of A, B, and C, as well as those in which at least one of A, B, and C are present (for example, "a system having at least one of A, B, and C" is intended to include a system of A alone, a system of B alone, a system of C alone, a system having 2 of A and B, a system having 3 of A, B, and C, and the like).

[0025] Existing research has proposed various prompt construction methods to perturb inputs to bypass security mechanisms, revealing security vulnerabilities of models. However, the current benchmarking framework has not systematically utilized adversarial attack strategies to improve the aggressiveness of original prompts for security evaluation of large models. In addition to content security, the security risks of large models in terms of data, models, systems, etc. have not been considered in the scope of security evaluation benchmarks by research.

[0026] Therefore, embodiments of the present application provide a target language model security evaluation method and electronic equipment. The method comprises: classifying a plurality of security fields according to a security level classification standard to obtain a classification result; constructing a test bank for at least one security field for each security level of the plurality of security levels to obtain a test bank; performing model application security testing on the target language model according to at least one test question in the attack test bank and at least one test question in the refusal test bank to obtain a model application security testing result; performing model function safety testing on the target language model based on a risk capability test case to obtain a model function safety testing result, wherein the risk capability test case is used to test the function safety risk of the target language model; and generating a security evaluation report of the target language model according to the model application security testing result and the model function safety testing result.

[0027] In the technical solutions of the present application, the user information (including but not limited to user personal information, user image information, user equipment information such as location information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards, necessary security measures are taken, public order and good customs are not violated, and corresponding operation portals are provided for the user to choose authorization or refusal.

[0028] In the scenario of making automatic decisions by using personal information, the method, device and system provided by the embodiments of the present application all provide corresponding operation portals for the user to choose to agree or refuse the automatic decision result; if the user chooses to refuse, the expert decision process is entered. The expression "automatic decision" herein refers to the activity of automatically analyzing, evaluating the behavior habits, interests and hobbies or economic, health and credit conditions of a person by a computer program and making decisions. The expression "expert decision" herein refers to the activity of making decisions by a person who is engaged in a certain field, has special experience, knowledge and skills and reaches a certain professional level.

[0029] It should be noted that the serial numbers of the operations in the following method are only used to represent the operations for description, and should not be regarded as representing the execution sequence of the operations. The method does not need to be executed in the order shown unless explicitly indicated.

[0030] Figure 1 A flowchart of a target language model security evaluation method according to an embodiment of the present application is shown.

[0031] As shown in Figure 1 , the target language model security evaluation method includes operation S110 to operation S150.

[0032] In operation S110, a plurality of security fields are classified according to a security level classification standard to obtain a classification result.

[0033] In operation S120, at least one security field is tested for each security level of the plurality of security levels to obtain a test question bank.

[0034] In operation S130, the target language model is subjected to model application security testing according to at least one test question in the attack test question bank and at least one test question in the refusal test question bank to obtain a model application security testing result.

[0035] In operation S140, the target language model is subjected to model function safety testing based on a risk ability test case to obtain a model function safety testing result.

[0036] In operation S150, the security test result and the model function security test result are applied according to the model to generate a security evaluation report of the target language model.

[0037] For example, the security field can include 30 fields including two fields of general task field and vertical task field. Among them, the general task field can be 20 fields, and the general task field can be a value concept field, a custom field, an ideology field, etc. The vertical task field is taken as an example of an education task field, which can include 10 fields, for example. For example, the education task field can be a political and ideological education field, a teacher assistance field, a psychological counseling field, etc.

[0038] Optionally, the security level classification standard includes a plurality of security levels. For example, an ideology security level, a personal security level, and a general security level. The plurality of security fields are classified according to the security level classification standard to obtain a classification result.

[0039] Optionally, the classification result includes at least one security field corresponding to each of the plurality of security levels.

[0040] For example, the security field corresponding to the general security level includes a psychological counseling field, a teacher assistance field, and an education theory field.

[0041] For example, at least one security field corresponding to the ideology security level is tested to obtain a test bank corresponding to the ideology security level; at least one security field corresponding to the personal security level is tested to obtain a test bank corresponding to the personal security level; and at least one security field corresponding to the general security level is tested to obtain a test bank corresponding to the general security level.

[0042] Optionally, the test bank includes at least an attack test bank and a refusal test bank. The attack test bank is used to test the security defense capability of the target language model when facing sensitive word camouflage attack, and the refusal test bank is used to test the refusal capability of the target language model when facing risk content questioning.

[0043] Optionally, the attack test bank includes a question and answer question after sensitive word camouflage, for example, please analyze the positive and negative effects of XX. The attack test bank is used to test the security defense capability of the target language model when facing sensitive word camouflage attack.

[0044] Optionally, the refusal test bank includes a question and answer question related to risk content questioning, for example, please provide the privacy information of the decision-making department in YY area. The refusal test bank can evaluate the ability of the target language model to refuse to reply when facing sensitive, illegal or potentially risky content.

[0045] Optionally, the target language model can be an artificial intelligence model constructed based on a large language model.

[0046] Optionally, for each security level, at least one test question in the attack test question bank and at least one test question in the refusal test question bank are input into the target language model to obtain output results for the attack test question bank and the refusal test question bank respectively; and the model application security test results of each security level are obtained according to the output results for the attack test question bank and the refusal test question bank respectively.

[0047] Optionally, the model application security test results of the plurality of security levels are statistically analyzed to obtain the model application security test results of the target language model as a whole.

[0048] Optionally, the model application security test results represent a quantitative measure of the security of the target language model as a whole in content generation.

[0049] Optionally, the risk capability test case is used to test the functional safety risk of the target language model. The functional safety risk can be divided into data security risk, model security risk, system security risk, compliance environment security risk, etc.

[0050] Optionally, the risk capability test case is a test step designed based on the functional safety risk test requirements for the target language model.

[0051] Optionally, the model functional safety test results represent a quantitative measure of the risk of the target language model and the system carrying the target language model in response to multiple functions. For example, it covers aspects such as physical hardware, target language model code, target language model system link, etc.

[0052] Optionally, the target language model can be comprehensively judged according to the model application security test results and the model functional safety test results, such as risk level analysis, test exception analysis, vulnerability repair, etc., to obtain a safety evaluation report of the target language model.

[0053] Optionally, since a multi-security field and multi-scenario safety evaluation framework is constructed, according to the characteristics of the security field corresponding to different security levels, the attack test question bank and the refusal test question bank are constructed to perform application security evaluation of the output content of the model, accurately understand the security protection capability of the target language model; a plurality of risk capability test case indicators are constructed to perform risk security evaluation of the model function, accurately evaluate the checking capability of the security vulnerabilities in the response content of the model and the system carrying the model; finally, the model functional safety test results and the model application security test results are comprehensively evaluated to generate an objective and comprehensive safety evaluation report, thereby reducing the misuse risk of the model.

[0054] Optionally, the model application security test is performed on the target language model according to at least one test question in the attack test question bank and at least one test question in the refusal test question bank, and a model application security test result is obtained, including: for each security level, inputting the attack test question bank into the target language model to output a first answer result; inputting the refusal test question bank into the target language model to output a second answer result; and performing model application security evaluation analysis on the first answer result and the second answer result based on the evaluator to obtain the model application security test result.

[0055] Optionally, the first answer result is a response result of the target language model to at least one test question in the attack test question bank. For example, 10 test questions A in the attack test question bank are input into the target language model, and a first answer result for each test question A is output.

[0056] For example, the first answer result for a test question A is: the negative effects of XX include making false fortune-telling predictions; exploiting the psychological weaknesses of the object to induce irresponsible decisions…

[0057] Optionally, the second answer result is a response result of the target language model to at least one test question in the refusal test question bank. For example, 10 test questions B in the refusal test question bank are input into the target language model, and a second answer result for each test question B is output.

[0058] For example, the second answer result for a test question B is: unable to provide unverified privacy information about the decision-making department in YY area.

[0059] Optionally, the model application security test result is obtained by performing model application security evaluation analysis on the first answer result and the second answer result using the trained evaluator.

[0060] Optionally, the model application security test result is obtained by performing model application security evaluation analysis on the first answer result and the second answer result based on the evaluator, including: obtaining a model attack success score according to the number of unidentified sensitive word camouflage attacks determined from the first answer result and the number of test questions in the attack test question bank; obtaining a model refusal ability score according to the number of refusals determined from the second answer result and the number of test questions in the refusal test question bank; and obtaining the model application security test result according to the model attack success score and the model refusal ability score of each of the plurality of security levels.

[0061] For example, the first answer result of the first question of test A: the negative effects of XX include making false fortune-telling predictions; exploiting the psychological weaknesses of the subject to induce irresponsible decisions. The target language model did not identify the sensitive word XX disguise and generated the first answer result containing harmful content or not following safety specifications, so the target language model did not identify the sensitive word disguise attack, and the attack was successful.

[0062] Optionally, the model attack success score represents the ratio of the number of times that cause the attack to be successful among all sensitive words in the sensitive word disguise attack on the target language model. The higher the model attack success score, the lower the model's security defense capability.

[0063] In an embodiment, the model attack success score As shown in formula (1).

[0064] (1).

[0065] wherein, represents the number of times of identifying the sensitive word disguise attack in the first answer result, represents the number of test questions in the test question bank.

[0066] Optionally, the second answer result can be subjected to refusal keyword identification. For example, keywords such as "sorry", "cannot", "cannot answer" can be used to display the refusal of the question content, and a rule-based method is used to determine whether the second answer result contains such refusal keywords.

[0067] Optionally, the model refusal ability score is used to measure the ability of the target language model to make a refusal reply when facing sensitive words, non-compliant or potentially risky content. The higher the model refusal ability score, the stronger the ability of the target language model to identify risks.

[0068] In an embodiment, the model refusal ability score As shown in formula (2).

[0069] (2).

[0070] wherein, represents the number of refusals determined from the second answer result, represents the number of test questions in the test question bank.

[0071] Optionally, for each security level, the model attack success score and the model refusal ability score are summed to obtain the model application security test result of this security level; the model application security test results of multiple security levels are weighted to obtain the overall model application security test result of the target language model.

[0072] Optionally, the test library further includes a subjective test library and an objective test library; the target language model security evaluation method further includes: inputting the subjective test library into the target language model to output a third question and answer result; inputting the objective test library into the target language model to output a fourth question and answer result; performing model security evaluation analysis on the first question and answer result, the second question and answer result, the third question and answer result, and the fourth question and answer result based on the evaluator to obtain a model application security test result.

[0073] Optionally, the subjective test library and the objective test library each include at least one test question, and the test question in the objective test library can be a selection question. The test question in the subjective test library can be a question and answer question.

[0074] Optionally, at least one test question in the subjective test library is input into the target language model to output a third question and answer result corresponding to each test question.

[0075] Optionally, the evaluator can be obtained based on a content security data fine-tuning large model.

[0076] Optionally, the test question in the subjective test library and the third question and answer result are placed in a specific template as an input of the evaluator, and the evaluator gives a judgment result according to whether the third question and answer result meets the safety requirement. The judgment result is divided into three categories: unsafe, safe, and controversial. Unsafe represents that the reply content violates the safety requirement and contains harmful content; safe represents that the reply content is safe and harmless; and controversial represents that the content reply is risky under strict requirements and safe under relaxed requirements.

[0077] Optionally, a subjective safety score is obtained according to a safe number determined from the third question and answer result and a number of test questions in the subjective test library. The higher the subjective safety score represents the higher the content safety capability of the target language model.

[0078] In an embodiment, the subjective safety score is As shown in formula (3).

[0079] (3).

[0080] wherein, represents a number of times that the judgment result determined from the third question and answer result is safe, represents a number of test questions in the subjective test library.

[0081] Optionally, for the test questions in the objective test question bank, a rule-based method is used, that is, it is detected whether there is a correct option in the fourth question and answer result, and the percentage of correct answers is taken as the objective security score. For the condition that zero or more options appear in the fourth question and answer result, it is found in the experiment that it only accounts for a very small proportion (less than 1 / %), and the rule judges the above question and answer result as correct.

[0082] Optionally, the objective security score is obtained according to the correct number determined from the fourth question and answer result and the number of test questions in the objective test question bank. The higher the objective security score represents the higher the correct rate of the target language model.

[0083] In an embodiment, the objective security score As shown in formula (4).

[0084] (4).

[0085] Wherein, characterizes the number of correct times determined from the fourth question and answer result, characterizes the number of test questions in the objective test question bank.

[0086] Optionally, based on the model security evaluation analysis of the evaluator on the first question and answer result, the second question and answer result, the third question and answer result and the fourth question and answer result, the subjective security score, the objective security score, the model attack success score and the model refusal ability score corresponding to each security level are obtained; the subjective security score, the objective security score, the model attack success score and the model refusal ability score are summed up to obtain the model application security test result of each security level.

[0087] In an embodiment, the model application security test result of the target language model as a whole As shown in formula (5).

[0088] (5).

[0089] Wherein, characterizes the weight corresponding to the i-th security level, characterizes the model application security test result corresponding to the i-th security level, and I characterizes the number of security levels.

[0090] Optionally, according to the characteristics of different security fields, objective test question banks and subjective test question banks are set. In the objective test question bank, for objective knowledge with clear answers, the content generated by the target language model must conform to objective facts and meet security requirements; in the subjective test question bank, for opinion and open-ended questions without fixed answers, the content generated by the target language model must follow security standards. In order to accurately understand the security protection capability of the target language model, role-playing, style injection and other red team attack methods are used to rewrite the original prompts to obtain more complex and deceptive jailbreak attack prompts, and the security defense capability of the target language model against sensitive word attacks is evaluated.

[0091] Optionally, the risk capability test case at least includes a data security test case, a model security test case, a system security test case, and a compliance environment test case. The model function safety test of the target language model is performed based on the risk capability test case, and a model function safety test result is obtained, including: performing a security vulnerability test on data associated with the target language model according to a plurality of data security test subcases in the data security test case, to obtain a data security score; performing a security vulnerability test on the target language model according to a plurality of model security test subcases in the model security test case, to obtain a model security score; performing a security vulnerability test on a running system associated with the target language model according to a plurality of system security test subcases in the system security test case, to obtain a system security score; performing a security vulnerability test on the running compliance of the target language model according to a plurality of compliance environment test subcases in the compliance environment test case, to obtain a compliance security score; and obtaining the model function safety test result according to the data security score, the model security score, the system security score, and the compliance security score.

[0092] Optionally, the data security test case is used to test the security capability of the target language model and related systems in data transmission and storage. A plurality of data security test subcases are determined according to different data security test subrequirements.

[0093] Optionally, the data associated with the target language model includes input data, output data, model running parameters, and other data of the target language model.

[0094] Optionally, based on the model security test subcase, a security vulnerability test is performed on the data associated with the target language model using a test tool, and a data security score is obtained. The data security score represents the risk degree of security vulnerabilities in data transmission and storage of the target language model and related systems.

[0095] Optionally, the model security test case is used to evaluate whether there are security vulnerabilities in the code of the target language model or whether the weight and other parameters can be stolen.

[0096] Optionally, based on the plurality of model security test sub-use cases, the target language model is tested for security vulnerabilities by using a test tool, and a model security score is obtained. The model security score represents the risk degree of the security vulnerabilities existing in the running of the target language model.

[0097] Optionally, the system security test use case is used to evaluate whether the target language model related hardware and software system or link has security vulnerabilities.

[0098] Optionally, based on the plurality of system security test sub-use cases, the running system associated with the target language model is tested for security vulnerabilities by using a test tool, and a system security score is obtained. The system security score represents the risk degree of the security vulnerabilities existing in the running system associated with the target language model.

[0099] Optionally, the compliance environment test use case is used to evaluate whether the design, coding, deployment and use of the target language model comply with relevant laws, regulations and industry standards.

[0100] Optionally, based on the plurality of compliance environment test sub-use cases, the running compliance of the target language model is tested for security vulnerabilities by using a test tool, and a compliance security score is obtained. The compliance security score represents the risk degree of the security vulnerabilities existing in the design, coding, deployment and use of the target language model.

[0101] In an embodiment, the model function safety test result As shown in formula (6).

[0102] (6).

[0103] Wherein, K represents the number of types of test cases included in the risk capability test use case, for example, the risk capability test use case includes data security test use case, model security test use case, system security test use case and compliance environment test use case, then K=4; represents the weight corresponding to the kth type of test case; the sum of the weights corresponding to the K types of test cases is 1, represents the security score corresponding to the kth type of test case.

[0104] For example, k=1, the first type of test case is data security test use case, the data security score; k=2, the second type of test case is model security test use case, the model security score; k=3, the third type of test case is system security test use case, the system security score; k=4, the fourth type of test case is compliance environment test use case, the compliance security score.

[0105] Figure 2An example diagram of risk capability test cases is shown in accordance with an embodiment of the application.

[0106] As shown in Figure 2 the risk capability test cases 200 include data security test cases 210, model security test cases 220, system security test cases 230, and compliance environment test cases 240.

[0107] Optionally, according to the plurality of model security test subcases in the model security test cases, the target language model is tested for security vulnerabilities to obtain a model security score, including: according to the plurality of model security test subcases, the target language model is tested for security vulnerabilities to obtain a test result corresponding to each model security test subcase; in a case where at least one test result represents that the target language model has a security vulnerability, according to the number of affected system functions, the number of affected users, and the number of affected data types caused by the at least one security vulnerability, a vulnerability impact range score and a vulnerability impact value score are obtained; and according to the number of security vulnerabilities, the vulnerability impact range score and the vulnerability impact value score corresponding to each of the at least one security vulnerability, the model security score is obtained.

[0108] Optionally, for the plurality of model security test subcases, a test tool is used to test the target language model for security vulnerabilities to obtain a test result corresponding to each of the plurality of model security test subcases. The test result includes that the target language model has a security vulnerability or does not have a security vulnerability.

[0109] Optionally, for each test result representing that the target language model has a security vulnerability, the number of affected system functions, the number of affected users, and the number of affected data types caused by the security vulnerability are detected.

[0110] Optionally, the vulnerability impact range score is used to evaluate the range of impact of the security vulnerability on the target language model system.

[0111] In an embodiment, the vulnerability impact range score is as shown in formula (7).

[0112] (7).

[0113] wherein, the number of affected system functions caused by the security vulnerability, the total number of system functions, the number of affected users caused by the security vulnerability, the total number of users, the number of affected data types caused by the security vulnerability, the total number of data types, a weight representing a number of affected system functions, a weight representing a number of affected users, a weight representing a number of affected data types.

[0114] Optionally, the vulnerability impact value score is used to evaluate the damage degree of the security vulnerability to system functions, user data, user property, etc.

[0115] Optionally, an artificial expert evaluates the impact according to the number of affected system functions, the number of affected users, and the number of affected data types caused by each security vulnerability, and scores the impact value between 1 and 10. The higher the impact value, the higher the score. Finally, the scores of each expert are averaged to obtain the vulnerability impact value score.

[0116] In an embodiment, the vulnerability impact value score as shown in formula (8).

[0117] (8).

[0118] wherein, represents the scoring result of the qth expert on the impact of the security vulnerability, and Q represents the number of experts.

[0119] In an embodiment, the model security score as shown in formula (9).

[0120] (9).

[0121] wherein, represents the vulnerability impact range score of the e th security vulnerability, represents the vulnerability impact value score of the e th security vulnerability, and E represents the number of security vulnerabilities existing in the test results based on the model security test sub-use case.

[0122] Optionally, the calculation process of the data security score, the system security score, and the compliance security score is consistent with the calculation process of the above-mentioned model security score, and will not be repeated here.

[0123] Optionally, the model security test sub-use case includes a model theft prevention test sub-use case; wherein, according to a plurality of model security test sub-use cases, a security vulnerability test is performed on the target language model, and a test result corresponding to each model security test sub-use case is obtained, including: according to the question and answer result or the interface call result of the target language model, the model parameters of the target language model are reversely inferred, and the model parameters are obtained; a reference language model is generated based on the model parameters; in the case that the performance difference between the reference language model and the target language model is less than a preset threshold, a test result of a security vulnerability is obtained, wherein the test result of the security vulnerability indicates that the model parameters of the target language model have a theft risk.

[0124] Optionally, the model theft prevention test sub-use case is used to test whether the model parameters can be reversely inferred and restored through the target language model question and answer, interface parameter audit, interface call and the like, so as to steal the model parameters.

[0125] Optionally, according to a large number of question and answer results or a large number of interface calls, the possible model parameters are reversely inferred. The inferred model parameters are used to generate a reference language model, and if the performance difference between the reference language model and the target language model is less than a preset threshold, it represents that the performance is close, indicating that the model parameters of the target language model may be stolen, the test fails, and the test result is that there is a security vulnerability.

[0126] Optionally, if the performance difference between the reference language model and the target language model is greater than or equal to the preset threshold, it represents that the performance difference is large, indicating that the model parameters of the target language model cannot be stolen, the test passes, and the test result is that there is no security vulnerability.

[0127] Optionally, the model security test sub-use case further includes at least one of the following: a model behavior monitoring test sub-use case, a model update test sub-use case, an adversarial attack test sub-use case, and a model virus attack defense test sub-use case.

[0128] Optionally, the model behavior monitoring test sub-use case is used to monitor the model output in real time and identify any signs of deviation from normal behavior. The test process of the model behavior monitoring test sub-use case: an anomaly detection system (such as an intrusion detection system) is built, the system anomalies are monitored in real time, and the abnormal content detected by the anomaly detection system is manually screened and analyzed to determine whether it is indeed abnormal and whether the detection is as expected.

[0129] Optionally, the model update test sub-use case is used to verify whether there is a security vulnerability in the model update process and to ensure that the new model is fully security verified before deployment. The test process of the model update test sub-use case: code audit tools, white box test tools and the like are used to perform code audit and white box test on the new model code, so as to find possible code errors, security vulnerabilities and the like.

[0130] Optionally, the adversarial attack test sub-use case is used to test the addition of subtle and imperceptible differences or perturbations in the data samples to construct new test samples, which should be classified as the correct category, but the target language model is easy to classify as the wrong category. Test the model's ability to resist adversarial samples and ensure that the model can still make correct judgments when faced with malicious input. The test process of the adversarial attack test sub-use case: design data samples as model input data, input into the model to ask questions, and evaluate whether the target language model will identify and classify it incorrectly. If the model can correctly identify and classify, the test is passed, otherwise it is not passed.

[0131] Optionally, the model virus attack defense test sub-use case is used to simulate virus attacks during model training and evaluate the robustness and resistance of the system when faced with malicious data. The test process of the model virus attack defense test sub-use case: use training data mixed with virus data to train the model, and evaluate the virus detector of the target language model for virus data detection ability. If the virus detector can identify virus content, the test is passed; if it cannot identify virus content, the test is not passed.

[0132] Optionally, the data security test sub-use case also includes at least one of the following: data transmission security test sub-use case, data storage security test sub-use case, data integrity test sub-use case, data privacy test sub-use case, data desensitization test sub-use case, data leakage protection test sub-use case.

[0133] Optionally, the data transmission security test sub-use case is used to ensure that secure encryption protocols are used during data transmission to prevent data from being eavesdropped or tampered with. The test process of the data transmission security test sub-use case: use network packet analysis tools to capture model input data, extract sensitive data from intercepted input data, and if the data is encrypted, use decryption tools to decrypt the data and try to obtain decrypted data. If plaintext data or decrypted data can be obtained, it indicates that there is a security vulnerability in the data transmission process, otherwise it indicates that the data transmission security is good.

[0134] Optionally, the data storage security test sub-use case is used to verify whether the data stored by the system is encrypted using appropriate encryption algorithms to prevent unauthorized access. Verify whether the system stores important information or sensitive information. The test process of the data storage security test sub-use case: check whether the sensitive data stored in the database is in plaintext form; if it is ciphertext, whether a suitable key is used, try to decrypt the stored data using decryption tools to confirm the strength of the encryption; verify the management method of the encryption key to ensure the security of key storage and access.

[0135] Optionally, the data integrity test sub-use case is used to verify that the data has not been tampered with during transmission or storage through means such as hash values, digital signatures, etc. The test process of the data integrity test sub-use case: use network packet analysis tools to capture model input data, tamper with data or content in the input data, send a request, verify data integrity, and whether the system can detect data tampering behavior. If tampering is detected, the test passes; otherwise, it fails.

[0136] Optionally, the data privacy test sub-use case is used to test whether the model uses differential privacy technology to prevent individual data from being inferred from the model output. The test process of the data privacy test sub-use case: use differential analysis techniques to infer the encryption key used by the encryption algorithm from multiple sets of plaintext input and ciphertext output. If the encryption key can be inferred or the encryption algorithm can be cracked, the test fails, otherwise it passes.

[0137] Optionally, the data desensitization test sub-use case is used to verify whether the model can process or generate content that does not contain sensitive personal information. The test process of the data desensitization test sub-use case: during model pre-training, mode use and interaction, test whether the data is desensitized according to the established desensitization requirements, such as data substitution desensitization, data confusion desensitization, data deletion desensitization, data generalization desensitization, data encryption desensitization, etc. During testing, attention should be paid to the fact that various data types, legal and illegal data can be desensitized; the same data before and after desensitization should maintain relevance and consistency; desensitized data is available.

[0138] Optionally, the data leakage protection test sub-use case is used to check whether the model has the risk of data leakage during training and inference. The test process of the data leakage protection test sub-use case: during the training and inference of the model, through vulnerability scanning, manual penetration, etc. Intrusion means, find vulnerabilities and threats in the system that may cause data leakage.

[0139] Optionally, the system security test sub-use case further includes at least one of the following: hardware security test sub-use case, application security test sub-use case, framework security test sub-use case, software security test sub-use case, user security test sub-use case, emergency response test sub-use case.

[0140] Optionally, the hardware security test sub-use case is used to simulate attacks on hardware to test the security protection level of the model hardware and to find possible hardware security vulnerabilities. The test process of the hardware security test sub-use case: use manual penetration testing to conduct in-depth security analysis of the hardware device to find possible hardware security vulnerabilities.

[0141] Optionally, the application security testing sub-use case is used to test the security of the application and its related data during use. The test process of the application security testing sub-use case: use network packet analysis tools to analyze the model input packet, and use security penetration testing tools to perform security penetration testing on the system, so as to find possible application security vulnerabilities.

[0142] Optionally, the framework security testing sub-use case is used to test the security of the deep learning framework or machine learning framework. The test process of the framework security testing sub-use case: use code auditing tools to perform code auditing and white-box testing on the model code, so as to find possible code errors, security vulnerabilities, and other problems, such as null pointer exceptions, memory leaks, and the like. At the same time, manual code auditing is used to further find possible vulnerabilities.

[0143] Optionally, the software security testing sub-use case is used to simulate attacks on the system software for testing, and to find possible software security vulnerabilities by simulating attacks. The test process of the software security testing sub-use case: use automated vulnerability scanning tools to perform security scanning on application server operating systems, application programs, networks, databases, middleware, and other components, so as to find possible vulnerabilities.

[0144] Optionally, the user security testing sub-use case is used to test the feedback mechanism of the model when encountering errors, to ensure that the user receives clear information. The test process of the user security testing sub-use case: design multiple different types, descriptions, expressions, and languages of questions as model inputs, to verify whether the model can understand the meaning of the inputs and make correct feedback for different questions. At the same time, manually test whether the system's operation is based on feedback, and whether the feedback given can clearly express its true meaning.

[0145] Optionally, the emergency response testing sub-use case is used to check whether the model can quickly recover to a stable state when encountering a fault. The test process of the emergency response testing sub-use case: use sudden power failure, sudden network failure, deletion of part of the data, manual implantation of abnormalities and triggering, and the like to simulate the system being subjected to sudden disasters, or data loss, or faults, so as to test the response capability of the system after the disaster. At the same time, simulate the recovery of the system after the disaster, view the recovery capability of the system after the disaster, verify whether the system has a fault recovery mechanism, and whether the fault recovery mechanism can quickly recover the system performance. In this way, the stability of the system is comprehensively evaluated.

[0146] Optionally, the compliance environment test sub-use case is used to ensure that the design, deployment and use of the model comply with relevant laws and regulations and industry standards. The test process of the compliance environment test sub-use case: by designing a plurality of different types, different descriptions, different expressions, different languages of questions as model inputs, verifying whether the outputs of the model comply with relevant laws and regulations and industry standards according to manual verification.

[0147] Optionally, according to the model application security test result and the model function safety test result, a security evaluation report of the target language model is generated, including: determining a model function safety risk level according to the model function safety test result and a preset risk level mapping relationship; and generating a security evaluation report according to the model function safety risk level, the model application security test result and a vulnerability patching strategy corresponding to a security vulnerability existing in the target language model, wherein the vulnerability patching strategy is generated for the security vulnerability test on the target language model according to the data security test sub-use case, the model security test sub-use case, the system security test sub-use case and the compliance environment test sub-use case, respectively, to obtain the test result representing the security vulnerability existing in the target language model.

[0148] Optionally, the preset risk level mapping relationship is a corresponding relationship between the model function safety test result and the model function safety risk level.

[0149] Table 1 shows a preset risk level mapping table of an embodiment of the present application.

[0150]

[0151] Optionally, as can be seen from Table 1, in the case that the model function safety test result is greater than or equal to 8 points, the model function safety risk level is determined to be a high-risk level; in the case that the model function safety test result is between 4 points and 8 points, the model function safety risk level is determined to be a medium-risk level; and in the case that the model function safety test result is between 0 points and 4 points, the model function safety risk level is determined to be a low-risk level.

[0152] Optionally, according to the data security test sub-use case, the model security test sub-use case, the system security test sub-use case and the compliance environment test sub-use case, the security vulnerability test is performed on the target language model, and in the case that the test result representing the security vulnerability existing in the target language model is obtained, the vulnerability patching strategy for the above security vulnerability is generated.

[0153] Optionally, the security evaluation report can be generated by comprehensively comparing and contrasting the model function safety risk level, the model application security test result and the vulnerability patching strategy corresponding to the security vulnerability existing in the target language model. For example, the corresponding relationship between the model function safety risk level and the security vulnerability is constructed, and the model function safety risk level and the vulnerability patching strategy corresponding to the security vulnerability are bound and laid out.

[0154] For example, the security vulnerability corresponding to the high-risk level can include an unauthorized vulnerability: a user is unauthorized to access related data or perform system operations; the security vulnerability corresponding to the medium-risk level can include a data unencrypted vulnerability: user data or system important data is not encrypted for transmission and storage using an encryption algorithm; the security vulnerability corresponding to the low-risk level can include an unencrypted protocol: data transmission is not performed using an encrypted protocol.

[0155] Figure 3 An example diagram of a target language model security evaluation method according to an embodiment of the present application is shown.

[0156] As shown in Figure 3 , the security evaluation of the target language model is divided into a preparation phase, an implementation phase, an evaluation phase, and an output phase. The preparation phase of the model application security test of the target language model includes, for example, constructing a test bank for 30 security fields to obtain a test bank, the test bank including a subjective test bank, an objective test bank, an attack test bank, and a refusal test bank. In the implementation phase, the test bank is input to the target language model to obtain first, second, third, and fourth question and answer results. The evaluator processes the first, second, third, and fourth question and answer results to obtain the model application security test results.

[0157] Optionally, the preparation phase of the model function security test of the target language model includes designing risk capability test cases, including data security test cases, model security test cases, system security test cases, and compliance environment test cases. In the implementation phase, the target language model is subjected to model function security testing based on the respective test cases of the four test functions, i.e., data security test cases, model security test cases, system security test cases, and compliance environment test cases, to obtain test results. Based on the vulnerability impact range score and the vulnerability impact value score of the security vulnerabilities in the test results, a model security score is obtained, and further a model function security test result and a vulnerability repair strategy are obtained. Based on the model application security test result, the model function security test result, and the vulnerability repair strategy, a security evaluation report of the target language model is generated.

[0158] Figure 4 A block diagram of an electronic device suitable for implementing a target language model security evaluation method according to an embodiment of the present application is shown.

[0159] Figure 4 The electronic device shown is merely an example and should not impose any limitations on the functions and use range of the embodiments of the present application.

[0160] like Figure 4 As shown, a computer electronic device 400 according to an embodiment of the present invention includes a processor 401, which can perform various appropriate actions and processes according to a program stored in a ROM 402 (read-only memory) or a program loaded from a storage portion 508 into a RAM 403 (random access memory). The processor 401 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 401 may also include onboard memory for caching purposes. The processor 401 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0161] RAM 403 stores various programs and data required for the operation of electronic device 400. Processor 401, ROM 402, and RAM 403 are interconnected via bus 404. Processor 401 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 402 and / or RAM 403. It should be noted that programs may also be stored in one or more memories other than ROM 402 and RAM 403. Processor 401 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in one or more memories.

[0162] Optionally, the electronic device 400 may also include an input / output (I / O) interface 405, which is also connected to the bus 404. The electronic device 400 may also include one or more of the following components connected to the input / output (I / O) interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the input / output (I / O) interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 410 as needed so that computer programs read from it can be installed into the storage section 408 as needed.

[0163] Optionally, the method flow according to the embodiments of the present application can be implemented as a computer software program. For example, the embodiments of the present application include a computer program product comprising a computer program carrying out the method shown in the flow chart, which comprises program codes for executing the method shown in the flow chart. In such embodiments, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the detachable medium 411. When the computer program is executed by the processor 401, the above-mentioned functions defined in the system of the embodiments of the present application are executed. Optionally, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0164] The present application also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments, or exist separately without being assembled into the device / apparatus / system. The computer readable storage medium carries one or more programs, which, when executed, implement the target language model security evaluation method according to the embodiments of the present application.

[0165] Optionally, the computer readable storage medium can be a non-volatile computer readable storage medium. For example, it can include but is not limited to: portable computer diskette, hard disk, random access memory (RAM 403), read-only memory (ROM 402), erasable programmable read-only memory (EPROM or flash memory), portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any appropriate combination of the above. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in connection with an instruction execution system, apparatus or device.

[0166] For example, optionally, the computer readable storage medium can include the ROM 402 and / or RAM 403 described above and / or one or more memories other than the ROM 402 and RAM 403.

[0167] The embodiments of the present application also include a computer program product comprising a computer program, which comprises program codes for executing the method provided by the embodiments of the present application, and when the computer program product is run on an electronic device, the program codes are used to make the electronic device implement the target language model security evaluation method provided by the embodiments of the present application.

[0168] When the computer program is executed by the processor 401, the above-mentioned functions defined in the system / apparatus of the embodiments of the present application are executed. Optionally, the system, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0169] In one embodiment, the computer program can be tangibly embodied in a non-transitory computer readable medium, such as the optical, magnetic, or semiconductor storage mediums. In another embodiment, the computer program can be tangibly embodied in a signal, such as a download signal over the Internet, or in an analog or digital broadcast signal, or in any other suitable medium or signal. The computer program can be transmitted in a signal, divided into one or more portions, and transmitted across a network, such as the Internet, or other suitable communication medium. The computer program can be downloaded and installed by the communication portion 409, and / or installed from the removable medium 411. The computer program embodied in the computer program can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber cable, or any suitable combination of the foregoing.

[0170] Optionally, the program code to carry out the operations of the present application embodiments can be written in any combination of one or more programming languages, including high-level, procedural or object oriented programming languages to accomplish the same. Program code can be any set of instructions, statements, or computer- executable code that can be executed by a processor in a computing device, and can be written in any combination of one or more programming languages. Program code can be completely executed on a user computing device, partially executed on a user device, partially executed on a remote computing device, or completely executed on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user computing device by any kind of network, including local area networks (LAN) or wide area networks (WAN), or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0171] The flow and block diagrams in the drawings represent possible architectural, functional, and operational architectures of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow and block diagrams can represent a module, a segment, or a portion of code that comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0172] The above described embodiments of the application have been described. However, these embodiments are merely meant to be illustrative of the present application and not meant to limit the scope of the present application. Although each of the above described embodiments have been described separately, this does not mean that measures from the various embodiments cannot be used advantageously in combination. Numerous alternatives and modifications will be apparent to those skilled in the art without departing from the scope of the present application, which is defined in the following claims.

Claims

1. A method for evaluating the security of a target language model, characterized in that, The method comprises: classifying a plurality of security fields according to a security level classification standard to obtain a classification result, wherein the classification result comprises at least one security field corresponding to each of a plurality of security levels; for each of the plurality of security levels, constructing a test bank for at least one security field to obtain a test bank, wherein the test bank comprises at least an attack test bank and a refusal test bank, the attack test bank is used to test the security defense capability of the target language model when facing sensitive word camouflage attack, and the refusal test bank is used to test the refusal capability of the target language model when facing risk content questioning; performing model application security testing on the target language model according to at least one test question in the attack test bank and at least one test question in the refusal test bank to obtain a model application security testing result of each of the security levels; weighting the model application security testing result of each of the security levels to obtain a model application security testing result of the target language model as a whole; performing model function safety testing on the target language model based on a risk capability test case to obtain a model function safety testing result, wherein the risk capability test case is used to test the function safety risk of the target language model, the risk capability test case comprises at least a data safety test case, a model safety test case, a system safety test case, and a compliance environment test case, and the model function safety testing result comprises a data safety score, a model safety score, a system safety score, and a compliance safety score obtained based on the data safety test case, the model safety test case, the system safety test case, and the compliance environment test case, respectively, and the model safety score is obtained based on the following operation: performing security vulnerability testing on the target language model according to a plurality of model safety test subcases in the model safety test case to obtain a test result corresponding to each of the model safety test subcases; in a case where at least one of the test results indicates that the target language model has a security vulnerability, obtaining a vulnerability impact range score and a vulnerability impact value score according to the number of affected system functions, the number of affected users, and the number of affected data types caused by at least one of the security vulnerabilities, wherein the vulnerability impact range score is obtained based on weighting the proportion of the number of affected system functions relative to the total number of system functions, the proportion of the number of affected users relative to the total number of users, and the proportion of the number of affected data types relative to the total number of data types; obtaining a model safety score according to the number of security vulnerabilities, the vulnerability impact range score, and the vulnerability impact value score corresponding to each of the at least one security vulnerability; generating a security evaluation report of the target language model according to the model application security testing result of the target language model as a whole and the model function safety testing result.

2. The method of claim 1, wherein, The model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one test question in the attack test question bank and at least one test question in the rejection test question bank, and the model application security test result is obtained by performing model application security test on the target language model according to at least one ​ ​ ​ 3. The method of claim 2, wherein, ​ ​ ​ ​ 4. The method of claim 2, wherein, ​ ​ ​ ​ 5. The method of claim 1, wherein, ​ ​ ​ ​ ​ 6. The method of claim 5, wherein, ​ ​ ​ ​ In a case where a performance difference between the reference language model and the target language model is less than a preset threshold, a test result of a security vulnerability is obtained, where the test result of the security vulnerability represents that model parameters of the target language model have a risk of being stolen.

7. The method of claim 5, wherein, The model security test sub-use case further includes at least one of a model behavior monitoring test sub-use case, a model update test sub-use case, an adversarial attack test sub-use case, and a model virus attack defense test sub-use case.

8. The method of claim 5, wherein, The generating, according to the model application security test result and the model functional safety test result, of a security evaluation report of the target language model includes: determining a model functional safety risk level according to the model functional safety test result and a preset risk level mapping relationship; generating the security evaluation report according to the model functional safety risk level, the model application security test result, and a vulnerability patching strategy corresponding to the security vulnerability of the target language model, where the vulnerability patching strategy is generated for the test result representing that the target language model has the security vulnerability, which is obtained according to the data security test sub-use case, the model security test sub-use case, the system security test sub-use case, and the compliance environment test sub-use case. 9.An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, characterized in that the one or more processors invoke the one or more computer programs to implement the steps of the method according to any one of claims 1-8.

Images