A safety evaluation system for large models

Abstract

The present invention discloses an evaluation system for the security of a large model, including a large model security evaluation module, which is used to perform a multi-dimensional security evaluation on a trained target large model to obtain a first evaluation result, and the multi-dimensionality includes robustness, bias and discrimination, inappropriate content, information leakage and information security; a data set evaluation module, which is used to perform a comprehensive evaluation on the training data set of the target large model to obtain a second evaluation result, and the comprehensive evaluation includes detecting the correctness of the labels of the training data set, whether there is toxic data, and the privacy leakage risk in the training data set; an evaluation report generation module, which is used to summarize the first evaluation result of the large model security evaluation module and the second evaluation result of the data set evaluation module, analyze the summary results and generate an evaluation report. The present invention focuses on solving the key problems in the current field of large model security evaluation, thereby comprehensively improving the security and stability of large models.

Description

Technical Field

[0001] The present invention relates to the technical field of large model evaluation, and in particular to an evaluation system for the safety of large models. Background Art

[0002] Currently, large language models based on the Transformer structure (referred to as large models) have become a hot topic and development trend in the market. However, the evaluation of their application has the problem of focusing on functional correctness and lacking multi-dimensional security evaluation, which cannot reveal the potential security risks of large models and their systems in complex application scenarios in advance. At the same time, the existing solutions do not effectively evaluate the training data set, and the quality of the training data directly affects the performance and security of the large model.

[0003] The details are as follows: 1. Insufficient dimensions for security assessment of large models: Existing technologies usually focus on limited basic assessment dimensions, such as functional accuracy. Although these dimensions can reflect the performance of large models in certain specific scenarios, these assessments are difficult to fully reveal potential security issues of large models in the context of the increasing complexity of the actual application of large models. Large models may introduce bias, discrimination, or misinformation when generating content. They have a wide range of applications, especially in key areas such as medicine, finance, and education. Any potential risks may have a significant impact.

[0004] 2. Unable to evaluate the training dataset of the large model: Currently, most large model evaluation work focuses on the performance evaluation of the large model itself, lacking the evaluation of its training dataset. However, the quality and diversity of the training dataset directly affect the performance and fairness of the large model. If the training dataset is biased, incomplete, or has security issues, the output of the large model may produce unreasonable or harmful results.

[0005] In response to the above problems, the existing technology has made certain improvements. For example, Chinese patent CN118964167A introduces a solution based on label matching. This solution obtains the test set by manual generation, web crawling or tool generation, and labels it manually or with a large model. Finally, the risk of the target large model is evaluated and an evaluation report is generated by matching the similarity between the target large model output and the auxiliary test results. However, the disadvantage of this solution is that it cannot solve the two key problems of incomplete evaluation dimensions and inability to evaluate training data sets, which limits its application in comprehensive security evaluation of large models. Summary of the invention

[0006] In view of the problems and shortcomings of the prior art, the present invention provides a large model security assessment system, which focuses on solving key problems in the current large model security assessment field, thereby comprehensively improving the security and stability of large models.

[0007] The technical solution adopted by the present invention to solve the above technical problems is to provide a large model safety evaluation system, which is characterized by including: A large model security assessment module, which is used to perform a multi-dimensional security assessment on the trained target large model to obtain a first assessment result, wherein the multi-dimensional assessment includes robustness, bias and discrimination, inappropriate content, information leakage, and information security; A data set evaluation module, which is used to comprehensively evaluate the training data set of the target large model to obtain a second evaluation result, wherein the comprehensive evaluation includes detecting the correctness of the labels of the training data set, whether there is toxic data, and the privacy leakage risk in the training data set; An evaluation report generation module is used to summarize the first evaluation results of the large model security evaluation module and the second evaluation results of the data set evaluation module, analyze the summary results and generate an evaluation report.

[0008] Preferably, the large model safety assessment module includes: A robustness evaluation unit, which is used to evaluate the stability and adaptability of the trained target large model under harsh or changing conditions, and output a robustness test result; A bias and discrimination evaluation unit, which is used to evaluate whether the trained target large model exhibits unfair bias or discrimination in the decision-making process, and output bias and discrimination detection results; A content inappropriateness evaluation unit, which is used to evaluate whether the content generated by the trained target large model contains harmful, inappropriate or illegal information, and output a content inappropriateness detection result; An information leakage evaluation unit, which is used to evaluate whether the trained target large model contains personal privacy information that leaks user privacy, and output an information leakage detection result; An information security evaluation unit is used to evaluate the protection capability of the trained target large model when facing malicious attacks and output information security detection results.

[0009] Preferably, the robustness evaluation unit is used to perform adversarial robustness testing, noise robustness testing, output robustness testing, uncertainty testing and jailbreak success rate testing; The adversarial robustness test is used to evaluate whether the target large model can resist malicious samples generated by adversarial attacks; The noise robustness test is used to evaluate the performance of the target large model after adding noise to the input data; The output robustness test is used to focus on the impact of small changes in input data on the output of the target large model; The uncertainty test is used to measure the response of the target macromodel when faced with incomplete or ambiguous data; The jailbreak success rate test is used to measure whether the target large model is easy to be broken through when it is attacked by bypassing.

[0010] Preferably, the bias and discrimination evaluation unit evaluates the target large model by identifying the bias of sensitive features, and the sensitive features include gender, race and age.

[0011] Preferably, the content inappropriateness evaluation unit evaluates the target large model through two indicators: toxicity value and inappropriate content ratio; The toxicity value is used to quantify the intensity of bad information in the output content of the target large model, where the bad information includes malicious, insulting and offensive speech; The inappropriate content ratio is used to evaluate the proportion of inappropriate content in the output content of the target large model, so as to reduce harmful information generated by the target large model.

[0012] Preferably, the personal privacy information identified by the information leakage assessment unit includes name, ID card, driver's license, gender, age, place of origin, ethnicity, education, occupation, contact information, communication records, address book, real-time location, home address, school address, and work address.

[0013] Preferably, the information security evaluation unit performs information security evaluation on the target large model through XSS vulnerability evaluation and injection attack.

[0014] Preferably, the data set evaluation module includes: A label correctness detection unit, which is used to verify the accuracy and consistency of the label of each sample in the training data set and output a label correctness detection result; a toxic data detection unit, which is used to evaluate whether there is harmful or toxic data in the training data set and output a toxic data detection result, wherein the toxic or harmful data includes bias, discrimination, false information and malicious behavior; A privacy data evaluation unit is used to identify and detect whether the samples in the training data set contain personal sensitive information that leaks user privacy, and output the privacy data detection result. The personal sensitive information includes personal identity information, health records and financial data.

[0015] Preferably, the evaluation report generating module includes: An evaluation result summary unit is used to integrate and quantitatively analyze the detection results of each of the large model security evaluation modules and the data set evaluation module to generate quantitative analysis results, and the quantitative analysis results are displayed in tables and charts; A risk analysis unit is configured to evaluate the security risks existing in the trained target large model and the training data set, wherein the security risks include the security issues existing in the robustness detection results, bias and discrimination detection results, content inappropriateness detection results, information leakage detection results, and information security detection results evaluated by the large model security evaluation module, and the security issues existing in the label correctness detection results, toxic data detection results, and privacy data detection results detected by the data set evaluation module, and to classify each of the security risks into risk levels, wherein the risk levels include high risk, medium risk, and low risk; The improvement suggestion unit proposes an optimization plan based on the evaluation results of the risk analysis unit, wherein the optimization plan includes adversarial training, diversified data adjustment, introduction of privacy protection technology, and optimization of large model output detection mechanism.

[0016] Preferably, each of the optimization plans includes short-term goals, medium-term goals and long-term goals.

[0017] Compared with the prior art, the technical solution of the embodiment of the present invention has the following beneficial effects: The present invention proposes an evaluation system for the security of a large model, including a large model security evaluation module, which is used to perform a multi-dimensional security evaluation on a trained target large model to obtain a first evaluation result, wherein the multi-dimensionality includes robustness, bias and discrimination, inappropriate content, information leakage and information security; a data set evaluation module, which is used to perform a comprehensive evaluation on the training data set of the target large model to obtain a second evaluation result, wherein the comprehensive evaluation includes detecting the correctness of the labels of the training data set, whether there is toxic data, and the privacy leakage risk in the training data set; an evaluation report generation module, which is used to summarize the first evaluation result of the large model security evaluation module and the second evaluation result of the data set evaluation module, analyze the summary results and generate an evaluation report. The present invention performs a multi-dimensional security evaluation on both the trained target large model and its training data set, thereby significantly improving the comprehensiveness and accuracy of the evaluation, ensuring the overall security and reliability of the large model, and providing a guarantee for its stable operation in practical applications.

[0018] Furthermore, the present invention conducts a multi-dimensional security assessment on the trained target large model, including the robustness, bias and discrimination, inappropriate content, information leakage and information security of the target large model, and evaluates the performance and risks of the target large model from the above four aspects to ensure its security and reliability.

[0019] Furthermore, the present invention conducts a multi-dimensional security assessment on the quality of the training data set of the target large model, including detecting the correctness of the labels of the training data set, whether there is toxic data, and the risk of privacy leakage in the training data set, so as to ensure the security and reliability of the training data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are some embodiments of the present invention, not all embodiments. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0021] Figure 1 It is a schematic diagram of the structure of a large model safety assessment system according to a preferred embodiment of the present invention.

[0022] Figure 2 It is a structural diagram of the large model security assessment module of the preferred embodiment of the present invention.

[0023] Figure 3 Schematic diagram of the structure of the data set evaluation module of the preferred embodiment of the present invention.

[0024] Figure 4 It is a structural diagram of the evaluation report generation module of a preferred embodiment of the present invention.

[0025] Figure 5 This is a workflow diagram of a large model safety assessment system according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0027] The technical solution of the present invention is described in detail below with reference to specific embodiments.

[0028] Based on the problems existing in the prior art, an embodiment of the present invention provides a large model security assessment system, which focuses on solving key problems in the current large model security assessment field, thereby comprehensively improving the security and stability of large models.

[0029] This embodiment provides a large model security assessment system. Figure 1 As shown, it includes a large model security evaluation module, a data set evaluation module and an evaluation report generation module. Among them: The large model security assessment module is used to conduct a multi-dimensional security assessment on the trained target large model to obtain the first assessment result. The multiple dimensions include robustness, bias and discrimination, inappropriate content, information leakage and information security.

[0030] The data set evaluation module is used to comprehensively evaluate the training data set of the target large model to obtain the second evaluation result. The comprehensive evaluation includes detecting the correctness of the labels of the training data set, whether there is toxic data, and the privacy leakage risk in the training data set.

[0031] An evaluation report generation module is used to summarize the first evaluation results of the large model security evaluation module and the second evaluation results of the data set evaluation module, analyze the summary results and generate an evaluation report.

[0032] like Figure 2 As shown in the figure, the large model security evaluation module includes a robustness evaluation unit, a robustness evaluation unit, a bias and discrimination evaluation unit, a content inappropriateness evaluation unit, an information leakage evaluation unit, and an information security evaluation unit. The large model security evaluation module conducts a comprehensive security evaluation on the trained target large model through each evaluation unit, and provides high-quality test data for the evaluation report generation module. Among them: The robustness evaluation unit is used to evaluate the stability and adaptability of the trained target large model under harsh or changing conditions and output the robustness test results. The robustness evaluation includes adversarial robustness test, noise robustness test, output robustness test, uncertainty test and jailbreak success rate test.

[0033] Among them, the adversarial robustness test is used to evaluate whether the target large model can resist malicious samples generated by adversarial attacks; the noise robustness test is used to evaluate the performance of the target large model after adding noise to the input data; the output robustness test is used to focus on the impact of small changes in input data on the output of the target large model; the uncertainty test is used to measure the response of the target large model when facing incomplete or ambiguous data; the jailbreak success rate test is used to measure whether the target large model is easy to be breached when it is attacked by bypass.

[0034] The bias and discrimination evaluation unit is used to evaluate whether the trained target large model shows unfair bias or discrimination in the decision-making process, and output the bias and discrimination detection results. Furthermore, the bias and discrimination evaluation unit evaluates the target large model by identifying the bias of sensitive features, including gender, race, and age.

[0035] The content inappropriateness evaluation unit is used to evaluate whether the content generated by the trained target large model contains harmful, inappropriate or illegal information, and output the content inappropriateness detection result. Furthermore, the content inappropriateness evaluation unit evaluates the target large model through two indicators: toxicity value and inappropriate content ratio.

[0036] Among them, the toxicity value is used to quantify the intensity of negative information in the output content of the target large model, which includes malicious, insulting and offensive remarks; the proportion of inappropriate content is used to evaluate the proportion of inappropriate content in the output content of the target large model, so as to reduce the harmful information generated by the target large model.

[0037] The information leakage evaluation unit is used to evaluate whether the trained target large model contains personal privacy information that leaks user privacy, and output the information leakage detection result. Furthermore, the personal privacy information identified by the information leakage evaluation unit includes name, ID card, driver's license, gender, age, place of origin, ethnicity, education, occupation, contact information, communication records, address book, real-time location, home address, school address, and work address.

[0038] The information security evaluation unit is used to evaluate the protection capability of the trained target large model in the face of malicious attacks and output the information security detection result. Further, the information security evaluation unit performs information security evaluation on the target large model through XSS vulnerability evaluation and injection attack.

[0039] like Figure 3 As shown in the figure, the dataset evaluation module includes a label correctness detection unit, a toxic data detection unit, and a privacy data evaluation unit. The dataset evaluation module performs security tests on the training dataset of the target large model through these three units, and provides high-quality test data for the evaluation report generation module. Among them: The label correctness detection unit is used to verify the accuracy and consistency of the label of each sample in the training data set and output the label correctness detection result.

[0040] The toxic data detection unit is used to evaluate whether there is harmful or toxic data in the training data set and output the toxic data detection result. Toxic or harmful data includes bias, discrimination, false information and malicious behavior.

[0041] The privacy data evaluation unit is used to identify and detect whether the samples in the training data set contain personal sensitive information that leaks the user's privacy, and output the privacy data detection results. Personal sensitive information includes personal identity information, health records, and financial data.

[0042] like Figure 4 As shown, the evaluation report generation module includes an evaluation result summary unit, a risk analysis unit, and an improvement suggestion unit. Among them: The evaluation result summary unit is used to integrate and quantitatively analyze the detection results of each large model security evaluation module and data set evaluation module, generate quantitative analysis results, and display the quantitative analysis results in tables and charts.

[0043] The risk analysis unit evaluates the security risks existing in the trained target large model and the training data set. The security risks include the security issues existing in the robustness detection results, bias and discrimination detection results, content inappropriateness detection results, information leakage detection results and information security detection results evaluated by the large model security evaluation module, as well as the security issues existing in the label correctness detection results, toxic data detection results and privacy data detection results detected by the data set evaluation module, and classifies each security risk into risk levels including high risk, medium risk and low risk.

[0044] The improvement suggestion unit proposes optimization plans based on the evaluation results of the risk analysis unit, including adversarial training, diversified data adjustment, introduction of privacy protection technology, and optimization of large model output detection mechanism. Each optimization plan includes short-term goals, medium-term goals, and long-term goals.

[0045] like Figure 5 As shown in the figure, the workflow of the large model safety assessment system is as follows: Create a task. The user manually or the system automatically creates a new task and imports the trained target large model and its training dataset. This can be achieved by local upload or API call.

[0046] To execute tasks, the system processes the imported target large model and training data set, and distributes them to the large model security assessment module and the data set assessment module to perform large model security testing and training data set security testing respectively.

[0047] Large model security assessment: conduct multi-dimensional security assessment on the trained target large model and obtain the first assessment results. This includes: Robustness evaluation evaluates the stability of the target large model in harsh environments, tests its ability to resist adversarial attacks, noise interference, slight input changes, incomplete or ambiguous data, and bypass attacks, and ensures the reliability of the target large model under different conditions.

[0048] Bias and discrimination assessment detects whether the target large model exhibits bias or discrimination in the decision-making process, including bias or discrimination against sensitive characteristics such as gender, race, and age, to ensure that the output of the target large model is fair and unbiased.

[0049] Content inappropriateness assessment evaluates whether the content generated by the target large model contains inappropriate, malicious or offensive information. By quantifying the toxicity value and the proportion of inappropriate content, it ensures that the output of the target large model meets ethical and legal requirements.

[0050] Information leakage assessment detects whether the target large model has the risk of leaking user personal information, ensuring that sensitive information is not exposed when processing user data and avoiding privacy leakage.

[0051] Information security assessment evaluates the protection capabilities of the target large model when facing malicious attacks, effectively prevents the target large model from being attacked maliciously during application, and ensures its information security.

[0052] Dataset evaluation: conduct a comprehensive evaluation of the training dataset of the target large model and obtain the second evaluation results. This includes: Label accuracy evaluation verifies whether the label of each sample in the training data set is accurate and complete, ensures that the label is consistent with the actual data, and avoids large model training deviations caused by label errors.

[0053] Data toxicity rate assessment detects whether there is harmful, biased or inappropriate content in the training data set, evaluates the impact of toxic data in the training data set on the training results of the target large model, and reduces unfair or harmful outputs.

[0054] Privacy data assessment: identifying and detecting whether the training data set contains samples that may leak personal sensitive information, ensuring that the training data set complies with privacy protection regulations.

[0055] The evaluation results are summarized by integrating and quantitatively analyzing the test results of each large model security evaluation module and data set evaluation module to form a comprehensive evaluation report and help developers understand the various evaluation indicators and their potential risks.

[0056] Risk analysis evaluates the security risks existing in the trained target large model and training data set, and assesses the possible safety, ethical, and compliance risks.

[0057] Improvement suggestions: Based on risk analysis, specific optimization suggestions are provided to help developers resolve security risks in target large models and training data sets, and improve the security and compliance of the system.

[0058] Generate an evaluation report. Automatically generate a detailed evaluation report that summarizes the test results of each large model security evaluation module and data set evaluation module, the evaluation result summary unit analysis results, risk analysis results and improvement suggestions, making it easier for developers to optimize the target large model and conduct security reviews.

[0059] The evaluation system for the security of a large model of an embodiment of the present invention includes a large model security evaluation module, which is used to perform a multi-dimensional security evaluation on a trained target large model to obtain a first evaluation result, wherein the multi-dimensionality includes robustness, bias and discrimination, inappropriate content, information leakage and information security; a data set evaluation module, which is used to perform a comprehensive evaluation on the training data set of the target large model to obtain a second evaluation result, wherein the comprehensive evaluation includes detecting the correctness of the labels of the training data set, whether there is toxic data, and the privacy leakage risk in the training data set; an evaluation report generation module, which is used to summarize the first evaluation result of the large model security evaluation module and the second evaluation result of the data set evaluation module, analyze the summary results and generate an evaluation report. The present invention performs a multi-dimensional security evaluation on both the trained target large model and its training data set, thereby significantly improving the comprehensiveness and accuracy of the evaluation, ensuring the overall security and reliability of the large model, and providing a guarantee for its stable operation in practical applications.

[0060] Furthermore, the present invention conducts a multi-dimensional security assessment on the trained target large model, including the robustness, bias and discrimination, inappropriate content, information leakage and information security of the target large model, and evaluates the performance and risks of the target large model from the above four aspects to ensure its security and reliability.

[0061] Furthermore, the present invention conducts a multi-dimensional security assessment on the quality of the training data set of the target large model, including detecting the correctness of the labels of the training data set, whether there is toxic data, and the risk of privacy leakage in the training data set, so as to ensure the security and reliability of the training data set.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A large model safety evaluation system, characterized in that: include: A large model security assessment module, which is used to perform a multi-dimensional security assessment on the trained target large model to obtain a first assessment result, wherein the multi-dimensional assessment includes robustness, bias and discrimination, inappropriate content, information leakage, and information security; A data set evaluation module, which is used to comprehensively evaluate the training data set of the target large model to obtain a second evaluation result, wherein the comprehensive evaluation includes detecting the correctness of the labels of the training data set, whether there is toxic data, and the privacy leakage risk in the training data set; An evaluation report generation module is used to summarize the first evaluation results of the large model security evaluation module and the second evaluation results of the data set evaluation module, analyze the summary results and generate an evaluation report.

2. A large model safety evaluation system as claimed in claim 1, characterized in that: The large model safety assessment module includes: A robustness evaluation unit, which is used to evaluate the stability and adaptability of the trained target large model under harsh or changing conditions, and output a robustness test result; A bias and discrimination evaluation unit, which is used to evaluate whether the trained target large model exhibits unfair bias or discrimination in the decision-making process, and output bias and discrimination detection results; A content inappropriateness evaluation unit, which is used to evaluate whether the content generated by the trained target large model contains harmful, inappropriate or illegal information, and output a content inappropriateness detection result; An information leakage evaluation unit, which is used to evaluate whether the trained target large model contains personal privacy information that leaks user privacy, and output an information leakage detection result; An information security evaluation unit is used to evaluate the protection capability of the trained target large model when facing malicious attacks and output information security detection results.

3. A large model safety evaluation system as claimed in claim 2, characterized in that: The robustness evaluation unit is used to perform adversarial robustness test, noise robustness test, output robustness test, uncertainty test and jailbreak success rate test; The adversarial robustness test is used to evaluate whether the target large model can resist malicious samples generated by adversarial attacks; The noise robustness test is used to evaluate the performance of the target large model after adding noise to the input data; The output robustness test is used to focus on the impact of small changes in input data on the output of the target large model; The uncertainty test is used to measure the response of the target macromodel when faced with incomplete or ambiguous data; The jailbreak success rate test is used to measure whether the target large model is easy to be broken through when it is attacked by bypassing.

4. A large model safety evaluation system as claimed in claim 2, characterized in that: The bias and discrimination evaluation unit evaluates the target large model by identifying the bias of sensitive features, and the sensitive features include gender, race and age.

5. A large model safety evaluation system as claimed in claim 2, characterized in that: The content inappropriateness evaluation unit evaluates the target large model through two indicators: toxicity value and inappropriate content ratio; The toxicity value is used to quantify the intensity of bad information in the output content of the target large model, where the bad information includes malicious, insulting and offensive speech; The inappropriate content ratio is used to evaluate the proportion of inappropriate content in the output content of the target large model, so as to reduce harmful information generated by the target large model.

6. A large model safety evaluation system as claimed in claim 2, characterized in that: The personal privacy information identified by the information leakage assessment unit includes name, ID card, driver's license, gender, age, place of origin, ethnicity, education, occupation, contact information, communication records, address book, real-time location, home address, school address, and work address.

7. A large model safety evaluation system as claimed in claim 2, characterized in that: The information security evaluation unit performs information security evaluation on the target large model through XSS vulnerability evaluation and injection attack.

8. A large model safety evaluation system as claimed in claim 1, characterized in that: The data set evaluation module includes: A label correctness detection unit, which is used to verify the accuracy and consistency of the label of each sample in the training data set and output a label correctness detection result; a toxic data detection unit, which is used to evaluate whether there is harmful or toxic data in the training data set and output a toxic data detection result, wherein the toxic or harmful data includes bias, discrimination, false information and malicious behavior; A privacy data evaluation unit is used to identify and detect whether the samples in the training data set contain personal sensitive information that leaks user privacy, and output the privacy data detection result. The personal sensitive information includes personal identity information, health records and financial data.

9. A large model safety evaluation system as claimed in claim 1, characterized in that: The evaluation report generation module includes: An evaluation result summary unit is used to integrate and quantitatively analyze the detection results of each of the large model security evaluation modules and the data set evaluation module to generate quantitative analysis results, and the quantitative analysis results are displayed in tables and charts; A risk analysis unit is configured to evaluate the security risks existing in the trained target large model and the training data set, wherein the security risks include the security issues existing in the robustness detection results, bias and discrimination detection results, content inappropriateness detection results, information leakage detection results, and information security detection results evaluated by the large model security evaluation module, and the security issues existing in the label correctness detection results, toxic data detection results, and privacy data detection results detected by the data set evaluation module, and to classify each of the security risks into risk levels, wherein the risk levels include high risk, medium risk, and low risk; The improvement suggestion unit proposes an optimization plan based on the evaluation results of the risk analysis unit, wherein the optimization plan includes adversarial training, diversified data adjustment, introduction of privacy protection technology, and optimization of large model output detection mechanism.

10. A large model safety evaluation system as claimed in claim 9, characterized in that: Each of the optimization plans includes short-term goals, mid-term goals and long-term goals.

Images