Security evaluation method and related device based on large language model

Abstract

The present application relates to the field of data processing, and in particular to a security assessment method based on a large language model and related devices. The method includes: identifying the security assessment dimensions matched by the large language model; generating autonomous test cases of the large language model based on the security assessment dimensions; the autonomous test cases include normal users and abnormal cases; the information content in the abnormal cases corresponds to the type to which the large language model belongs; inputting the autonomous test cases into the large language model to execute the model function, and obtaining the response results of the large language model to the autonomous test cases; and generating a security assessment report for the large language model based on the response results. The method uses autonomous test cases to actively evaluate the security of the model, realizes the automated and comprehensive evaluation of the model, improves the efficiency and accuracy of the model security assessment, enhances the adaptability to models of different fields and scales, realizes the automation and continuous optimization of security assessment, and ensures the security of model operation.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a security assessment method based on a large language model and related devices. Background Art

[0002] With the development of artificial intelligence technology, especially the application of deep learning and neural network technology, large language models are playing an increasingly important role in various fields of people's lives. This makes it extremely important to ensure the security and controllability of large language models.

[0003] Traditional evaluation relies on manually designed attack scenarios and audit results, and its efficiency and adaptability are worrying. When manually designing attack scenarios, it is difficult to cover all the security risks that the model may encounter when applied in multiple complex fields (such as psychological counseling, psychology, finance, law, etc.) due to the limitations of experience and knowledge. In the context of multiple fields intertwined, the complexity of professional terminology and rule logic makes it very easy for manual design to miss a single thing. The results of manual audits are even more difficult. Faced with the massive output data of the model, auditors are prone to fatigue and negligence, and it is difficult to identify security risks in a timely and accurate manner. For example, in the evaluation of social media content generation models, the amount of text generated by millions of texts every day far exceeds the manual audit load, resulting in potential security issues being ignored, which may in turn induce social and legal crises.

[0004] The existing evaluation methods for large-scale models have also become ineffective when faced with large-parameter models. The existing models have simple structures and few parameters, and the indicators and strategies on which their evaluation methods are based are stretched in the face of today's large language models with billions, tens of billions, and hundreds of billions of parameters. The neural network structure of large language models is complex, and the scale of training data is huge and diversely distributed. It is difficult for existing evaluation methods to deeply explore their complex internal data processing mechanisms and logic. For example, old methods based on simple statistical features or small-scale sample tests cannot detect deep-seated security risks such as information fusion and semantic understanding bias when large models process large-scale multimodal data, because they do not take into account the high complexity of large models and the subtle changes in large-scale data processing.

[0005] Therefore, it is urgent to propose a new technical solution to solve at least one of the technical problems of low efficiency, limited model parameters, and security vulnerabilities caused by manual review in the above-mentioned traditional evaluation methods. Summary of the invention

[0006] In response to the technical problems existing in the prior art, the present application provides a security assessment method and related devices based on a large language model to solve at least one of the technical problems of low efficiency, limited model parameters, and security vulnerabilities caused by manual review in the above-mentioned traditional assessment method, so as to adapt to the continuous expansion of model complexity and application fields and ensure the security and reliability of model application.

[0007] In a first aspect, an embodiment of the present application provides a security assessment method based on a large language model, the method comprising:

[0008] For the target model to be evaluated, identify the security assessment dimension matched by the target model; the target model is a large language model;

[0009] Generate autonomous test cases of the target model based on the security assessment dimension; the autonomous test cases include normal users and abnormal cases; the information content in the abnormal cases corresponds to the type of the target model;

[0010] Inputting the autonomous test case into the target model to execute the model function, and obtaining the response result of the target model to the autonomous test case;

[0011] A security assessment report for the target model is generated based on the response result.

[0012] In a second aspect, an embodiment of the present application provides a security assessment device based on a large language model, the device comprising at least the following units:

[0013] The identification unit is configured to identify, for a target model to be evaluated, a security assessment dimension matched by the target model; the target model is a large language model;

[0014] A generating unit is configured to generate an autonomous test case of a target model based on the security assessment dimension; the autonomous test case includes a normal user and an abnormal case; the information content in the abnormal case corresponds to the type of the target model;

[0015] An evaluation unit is configured to input the autonomous test case into the target model to execute the model function and obtain a response result of the target model to the autonomous test case;

[0016] The reporting unit is configured to generate a security assessment report for the target model based on the response result. In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0017] at least one processor, memory, and input-output unit;

[0018] The memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the security assessment method based on a large language model of the first aspect.

[0019] In a fourth aspect, a computer-readable storage medium is provided, which includes instructions. When the instructions are executed on a computer, the computer executes the security assessment method based on a large language model of the first aspect.

[0020] The beneficial effects of the present application are: providing a security assessment method based on a large language model and related devices. In this technical solution, first, for the target model to be evaluated, the security assessment dimension matched by the target model is identified; the target model is a large language model. Then, based on the security assessment dimension, an autonomous test case of the target model is generated; the autonomous test case includes normal users and abnormal cases; the information content in the abnormal case corresponds to the type to which the target model belongs. Then, the autonomous test case is input into the target model to execute the model function, and the response result of the target model to the autonomous test case is obtained. Finally, a security assessment report for the target model is generated based on the response result. The technical solution of the present application, by constructing a security assessment dimension and generating an adaptive autonomous test case to actively evaluate the model security, realizes the automatic and comprehensive evaluation of the model, improves the efficiency and accuracy of the model security assessment, enhances the adaptability to models of different fields and scales, realizes the automation and continuous optimization of security assessment, and ensures the security of model operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flowchart of a security assessment method based on a large language model according to an embodiment of the present application;

[0022] Figure 2 It is a structural schematic diagram of a security assessment device based on a large language model in an embodiment of the present application;

[0023] Figure 3 is a schematic diagram of the structure of an electronic device according to an embodiment of the present application;

[0024] Figure 4 It is a structural schematic diagram of a medium in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0026] With the rapid development of artificial intelligence technology, dialogue systems based on large language models have penetrated into people's daily lives and work. In particular, significant progress has been made in the research and application of task-based dialogue systems. Among related technologies, the security evaluation of large language model systems still faces many difficulties and shortcomings. They are mainly reflected in the following aspects.

[0027] First, traditional large language model security assessments are highly dependent on manually designed attack scenarios and audit results. When designing attack scenarios, humans need to rely on experience and limited knowledge reserves to conceive various scenarios that may cause model security vulnerabilities to be exposed. This process is extremely time-consuming and highly subjective. For example, for a large language model involving multi-field knowledge applications, it is difficult for humans to exhaust all scenarios where security risks may occur. For example, in professional fields such as psychology, finance, and law, different terms, rules, and logical relationships are intricate. Manually designed attack scenarios may only be the tip of the iceberg, and it is impossible to fully detect the security of the model when it is applied in these complex fields.

[0028] In terms of audit results, manual audits are not only slow, but also prone to fatigue and negligence. Faced with a massive amount of model output data, manual auditors need to identify whether there are security risks one by one, such as harmful information, privacy leaks, data bias, etc. For example, in the evaluation of social media content generation models, millions of text contents may be generated every day. It is difficult for manual auditors to promptly and accurately determine whether each content meets security standards, resulting in some potential security issues being missed, which may in turn cause serious social impacts or legal disputes.

[0029] Second, as large language models continue to develop, the number of their parameters has shown a trend of significantly increasing, which makes the security attacks and evaluation strategies previously built for machine learning or deep learning models no longer applicable. Existing evaluation methods developed for smaller models are inadequate when faced with large language models with an exponentially growing number of parameters. Existing model structures are relatively simple, with a small number of parameters, and their data processing flow and logic are relatively easy to understand and analyze. However, large language models with billions, tens of billions, or even hundreds of billions of parameters have more complex neural network structures and data interaction patterns.

[0030] The training data of these large models is huge, and the data distribution is more complex and diverse. The indicators and strategies used by existing evaluation methods are difficult to accurately measure their security. For example, some existing evaluation methods based on simple statistical features or small-scale sample tests cannot effectively capture deep-level security issues in the face of large language models, such as information fusion errors and security risks caused by semantic understanding bias when the model processes large-scale multimodal data, because these methods do not take into account the complex internal mechanisms of large models and various subtle changes in the process of large-scale data processing.

[0031] Third, many existing evaluation methods focus on checking the output results of the model, while ignoring the potential security risks during the operation of the model. Focusing only on the results will lead to incomplete detection of security risks. For example, in the evaluation of a large language model for intelligent customer service, if you only check whether the final answer to the user contains harmful information, without paying attention to how the model handles user data in the process of generating the answer, you may ignore the risk of privacy leakage that may occur when the data flows within the model.

[0032] During the training and reasoning process, the model may accumulate potential security risks due to algorithm defects, data pollution or inappropriate optimization strategies. These risks will only be reflected in the results under specific input conditions or after a long period of operation. However, due to the focus on result evaluation, the risks in these processes cannot be discovered and prevented in time, which makes the model face unpredictable security threats in actual applications. Once an outbreak occurs, it may cause serious damage to user rights, corporate reputation and social stability.

[0033] Therefore, it is urgent to propose a new technical solution to solve at least one of the technical problems of low efficiency, limited model parameters, and security vulnerabilities caused by manual review in the above-mentioned traditional evaluation methods.

[0034] In order to solve at least one technical problem in the related art, the embodiment of the present application provides a security assessment method based on a large language model and a related device. In this technical solution, first, for the target model to be evaluated, the security assessment dimension matched by the target model is identified; the target model is a large language model. Then, an autonomous test case of the target model is generated based on the security assessment dimension; the autonomous test case contains normal users and abnormal cases; the information content in the abnormal case corresponds to the type of the target model. Then, the autonomous test case is input into the target model to execute the model function, and the response result of the target model to the autonomous test case is obtained. Finally, a security assessment report for the target model is generated based on the response result. The technical solution of the present application, by constructing a security assessment dimension and generating an adaptive autonomous test case to actively evaluate the model security, realizes the automatic and comprehensive evaluation of the model, improves the efficiency and accuracy of the model security assessment, enhances the adaptability to models of different fields and scales, realizes the automation and continuous optimization of security assessment, and ensures the security of model operation.

[0035] The technical solution of the present application and the security assessment solution based on a large language model provided in the embodiment of the present application can also be executed by an electronic device, which can be a server, a server cluster, or a cloud server. The electronic device can also be a terminal device such as a mobile phone, a computer, a tablet computer, a wearable device, or a dedicated device (such as a dedicated terminal device with a security assessment method system based on a large language model, etc.). These electronic devices can also be equipped with the chips introduced in the above embodiments. Alternatively, these electronic devices can also be installed with a service program for executing a security assessment solution based on a large language model.

[0036] Figure 1 A flowchart of a security assessment method based on a large language model provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method comprises the following steps:

[0037] 101, for a target model to be evaluated, identifying a security evaluation dimension matched by the target model;

[0038] 102, generating autonomous test cases of a target model based on the security assessment dimension;

[0039] 103, inputting the autonomous test case into the target model to execute the model function, and obtaining the response result of the target model to the autonomous test case;

[0040] 104. Generate a security assessment report for the target model based on the response result.

[0041] In the embodiment of the present application, the target model is a large language model. Large language models usually have a large number of parameters, such as billions or even tens of billions or hundreds of billions of parameters, and have powerful language understanding and generation capabilities. Large language models are trained based on large-scale text data and can master rich language knowledge, semantic relationships, and concepts in different fields, so that they can flexibly respond to a variety of natural language processing tasks, such as text generation, question answering, translation, summarization, etc.

[0042] It is understandable that in the above steps, by specifically identifying the matching security assessment dimensions for the target large language model, it is possible to get rid of the limitations of a general but potentially inaccurate assessment framework. Since different large language models differ in application areas, data usage, model architecture, etc., for example, the security concerns of large language models in the psychological field are completely different from those in the financial field. Accurate identification can ensure that subsequent evaluation work is closely centered around the characteristics of the model itself. For example, for dialogue models, the focus can be on dimensions such as user privacy protection and accuracy of psychological counseling knowledge, laying a solid foundation for a comprehensive and in-depth assessment of model security, avoiding wasting resources on irrelevant assessment aspects, and improving evaluation efficiency and effectiveness.

[0043] In the embodiment of the present application, the autonomous test case includes normal users and abnormal cases. Further, the information content in the abnormal case corresponds to the type to which the target model belongs.

[0044] The autonomous test cases generated in the above steps include both normal user cases and abnormal cases, which is a very advantageous design. Normal cases can be used to verify the functional integrity and security of the model in normal usage scenarios, ensuring that the model can run stably and safely in normal business processes. For example, in the e-commerce recommendation large language model, normal cases can test whether its processing of regular user browsing and purchasing behavior data is accurate and in compliance with security specifications.

[0045] It is worth noting that abnormal use cases are customized according to the type of the target model, and can actively detect the performance of the model in edge cases and potentially dangerous scenarios. Taking the content generation model as an example, abnormal use cases may include intentionally inputting information that is biased, misleading, or violates laws and regulations, thereby effectively detecting whether the model has the ability to resist such bad inputs, preventing the model from being maliciously used in actual applications to produce harmful outputs, and greatly enhancing the comprehensiveness of the model security assessment.

[0046] In the above steps, the autonomous test cases are input into the target model and its response results are obtained to generate a security assessment report. This process can truly reflect the security status of the model during actual operation. Different from the evaluation method based only on theoretical analysis or simulation estimation, it directly faces the real reaction of the model when processing various inputs, whether it is the output accuracy and efficiency under normal input, or the error handling and security vulnerability exposure caused by abnormal input, which can be recorded and analyzed in detail.

[0047] The security assessment report generated in the above steps provides valuable data basis for subsequent decisions such as model optimization, deployment, and monitoring. The detailed information in the report can clearly indicate in which security assessment dimensions the model is weak. For example, if a model is found to have high-risk vulnerabilities in the privacy protection dimension, relevant personnel can formulate targeted improvement strategies based on this, such as optimizing data encryption algorithms, adjusting data access permissions, etc. At the same time, for model deployment decisions, the report can be used to determine whether additional security protection measures need to be added or the scope of application of the model needs to be limited, thereby ensuring that the security of the model is under control throughout its life cycle.

[0048] The specific implementation of steps 101 to 104 is described below with reference to a specific example.

[0049] As an optional embodiment, in 101, for the target model to be evaluated, identifying the security assessment dimension matched by the target model includes:

[0050] Perform model characteristic analysis on the target model to obtain model characteristic information of the target model; generate security assessment dimensions for comprehensive evaluation of the target model based on the model characteristic information and pre-set security assessment standards; use an expert system to verify and optimize the generated security assessment dimensions to obtain the final output security assessment dimensions.

[0051] In the embodiment of the present application, the model characteristic information at least includes: application scenario of the target model, model structure, and training data. Further, the application scenario at least includes: model function of the target model and the technical field to which the model belongs.

[0052] In security assessment based on large language models, model feature information is a key element to fully and deeply understand the target model and accurately determine the security assessment dimensions. It covers multiple aspects, mainly including the application scenarios, model structure and training data of the target model. Each aspect contains more specific sub-contents. These information are interrelated and together outline the overall image of the target model, providing a basis for the generation of subsequent security assessment dimensions.

[0053] Model functions refer to specific tasks or operations that a large language model can achieve, such as text generation (such as creating news articles, writing stories, etc.), intelligent question and answer (answering users' questions about knowledge in various fields), text translation, text summary extraction, etc. Taking a large language model of an intelligent writing assistant as an example, its main function is to generate corresponding high-quality text content based on the topics and requirements provided by users. This functional feature determines that the accuracy and compliance of the generated content should be focused on during security assessment.

[0054] The technical field to which the model belongs indicates the industry or professional scope in which the model is applied. Common ones include psychology (assisting doctors in analyzing medical records, providing psychological advice, etc.), finance (conducting investment analysis, risk prediction, answering customer inquiries, etc.), and education (assisting teaching, correcting homework, generating learning materials, etc.). For example, for large language models in the field of psychology, it is important to consider the security of handling patient privacy data and the reliability of psychological knowledge output, because it involves sensitive patient information and professional advice about health.

[0055] Model structure, including the model architecture type (such as the model architecture and its specific number of layers, heads and other parameter details), parameter scale (millions, tens of millions, billions or even more parameters), neuron connection method, etc. For example, the Transformer architecture has advantages in processing long sequences of text due to its unique self-attention mechanism, but it may also have specific security risks due to complex parameter interactions. A large language model with billions of parameters has a more complex internal data flow and processing mode than a small parameter model. When conducting security assessments, it is necessary to consider the impact of this complexity on the robustness of the model and other aspects.

[0056] Training data includes the source of the data (whether it comes from public Internet text, professional industry databases, or data compiled within the company, etc.), the size of the data (the number of texts in the training corpus, the magnitude of the data, etc.), the preprocessing method of the data (such as whether it has been cleaned to remove noise and duplicate content, whether it has been labeled, etc.), and the distribution characteristics of the data (the proportion of data in different topics and fields, etc.). For a general large language model trained with public Internet text, its training data comes from a wide range of sources but may be of uneven quality and contain some erroneous or biased information. This requires attention in security assessments to whether the model will output bad content due to these problematic data, and also to examine its handling of data privacy. Although the source is public, it may involve consideration of the original author's copyright and other related rights.

[0057] Through detailed analysis of these model characteristic information and combining it with pre-set security assessment standards, we can more scientifically and reasonably generate security assessment dimensions that fit the actual situation of the target large language model. After verification and optimization by the expert system, we can ensure the accuracy and effectiveness of the assessment dimensions, laying a solid foundation for subsequent comprehensive security assessment work.

[0058] In the embodiment of the present application, the expert system is obtained by adaptively configuring the security assessment knowledge graph based on the technical field to which the model belongs. Taking the large language model in the field of psychology as an example, a corresponding security assessment knowledge graph will be constructed for this field, and the graph integrates many knowledge elements related to the security assessment of the large language model of psychology. Like the classification standards of psychological data, which data are absolutely private (such as the patient's genetic information, specific details of the diagnosis and treatment process, etc.), and which are relatively public data; there are also indicators for measuring the accuracy of psychological diagnosis, references for the diagnosis of different diseases, and common misdiagnosis factors; as well as various potential security threats faced by the model in psychological scenarios, such as psychological advice tampering that may be caused by external attacks, and ways for patients' privacy data to be illegally obtained. Based on such a knowledge graph, the expert system can be configured adaptively. It can simulate the thinking and judgment of experts based on the association and logical relationship between different knowledge nodes in the graph. When the generated security assessment dimensions are to be verified and optimized, it can refer to the detailed definition of various aspects of the safety of the large language model of psychology in the knowledge graph to review whether the proposed assessment dimensions fully cover the key security points. For example, for the "Patient Privacy Data Protection Assessment" dimension, it can determine whether all possible links where privacy leaks may occur have been taken into account based on the relevant knowledge of psychological data privacy in the knowledge graph, such as whether the data transmission process, storage method, and access control mechanism when the model uses data are included in the assessment scope.

[0059] It is understandable that for large language models in other fields, the security assessment knowledge graphs in different fields will focus on the characteristics and security needs of their respective industries. For example, the knowledge graph in the financial field will build knowledge nodes and relationship networks around the security of financial transaction data, the reliability of investment risk prediction, and compliance. Based on this, the expert system can accurately judge whether the security assessment dimensions generated for the financial large language model are reasonable and complete, and ensure that the assessment dimensions are closely aligned with the actual security conditions faced by the model in this field during the verification and optimization process, thereby laying a solid foundation for subsequent accurate and effective security assessment work, making the entire assessment process more professional and targeted, and better able to discover and respond to potential security risks in the model.

[0060] For example, in a psychological counseling scenario, it is assumed that there is a large language model specifically used for online psychological counseling services. In the above embodiment, the model characteristic analysis is first performed to obtain the model characteristic information. In terms of application scenarios, the function of the model is to interact with the consultant, understand the consultant's emotions and psychological distress, and give corresponding psychological counseling suggestions and possible solutions. The technical field is the mental health counseling industry. This means that it will process a large amount of personal privacy information of the consultant, including sensitive content such as deep emotional problems and family conflicts, so the security assessment needs to focus on the confidentiality of the consultation content and the dimensions of the suitability and effectiveness of the suggestions. From the perspective of the model structure, it is assumed that an improved recurrent neural network architecture is adopted with a certain scale of parameters. This structure affects its ability to understand semantics and analyze sentiment. When conducting security assessments, its stability and accuracy in processing complex emotional expressions and semantically ambiguous texts should be considered, such as whether the consultant's intentions will be misinterpreted due to the limitations of the model structure, and then misguided guidance will be given. In terms of training data, the source may be a large number of public psychological counseling cases, relevant psychological research literature, and real counseling records that have been partially anonymized. The scale and quality of the data will affect the professional level and reliability of the model. For example, if there are incorrect analyses or inappropriate handling of certain psychological problems in the data, it may lead to deviations in the model output. Therefore, the evaluation dimension must include considerations of the quality and deviation of training data.

[0061] Then, based on the model characteristic information and pre-set security assessment standards such as the safety specifications of the psychological counseling industry and general network security guidelines, security assessment dimensions such as "consultation content privacy protection assessment", "psychological advice accuracy assessment", "model's emotional semantic understanding ability assessment", and "training data quality and bias assessment" will be generated.

[0062] Then, an expert system based on the adaptive configuration of the psychological counseling security assessment knowledge graph is used to verify and optimize these dimensions. The knowledge graph in the expert system integrates the classification of psychological counseling data (such as which information is absolutely confidential and which can be shared in a limited manner), diagnostic standards for different psychological problems, the correct application of common psychological intervention methods, and various potential security threats (such as the way of consulting information leakage, the situation where the model is maliciously guided to give harmful suggestions). The expert system reviews and evaluates the dimensions based on this. For example, for the "consulting content privacy protection assessment", it will determine whether the privacy protection considerations of data storage, transmission, and use are covered based on the knowledge graph. For the "psychological advice accuracy assessment", it will check whether the accuracy measurement under different types of psychological problems and individual differences of consultants is taken into account. After verification and optimization, the final output security assessment dimension can accurately fit the actual situation of the psychological counseling large language model and effectively detect potential security risks. Whether it is privacy leakage, misguidance, or output deviation caused by data problems, the risks can be accurately assessed, providing strong support for the safe use and continuous improvement of the model, and protecting the rights and interests of consultants and the quality of psychological counseling services.

[0063] Further optionally, in the above steps, before identifying the security assessment dimensions that match the target model to be evaluated, it is also possible to obtain security assessment standards for multiple technical fields; perform keyword extraction processing on the security assessment standards to obtain security assessment keywords corresponding to each of the multiple technical fields; wherein the keyword extraction processing at least includes: semantic recognition, semantic retrieval; using the security assessment keywords to mark the security assessment standards corresponding to each of the multiple technical fields. Furthermore, in the above steps, based on the model characteristic information and the pre-set security assessment standards, a security assessment dimension for comprehensively evaluating the target model is generated, including: based on the model characteristic information and the security assessment keywords, a security assessment dimension for comprehensively evaluating the target model is generated through a clustering algorithm.

[0064] Specifically, in the entire security assessment process, adding steps such as obtaining security assessment standards in multiple technical fields and extracting and annotating keywords and generating assessment dimensions based on them has many important effects.

[0065] First, obtaining security assessment standards in multiple technical fields broadens the reference scope. It is no longer limited to a single general standard, but comprehensively considers the regulatory requirements of different industry characteristics, such as the strict requirements for patient privacy protection in the psychological field and the focus on data compliance and risk prevention and control in the financial field. These can be included, so that the subsequent security assessment can be more in line with the actual application scenario and fully consider the unique security needs of the field where the target model is located. Alternatively, the security assessment standard can also be a security assessment standard for neural network models. For example, security assessment standards in various technical fields can be collected and sorted in advance, such as OWASP Top 10, NIST Cybersecurity Framework, etc. Next, keyword extraction processing is performed, and security assessment keywords are mined through semantic recognition and semantic retrieval. This can accurately extract the key core content of each standard and simplify the complex assessment standards into key conceptual expressions. For example, keywords such as "transaction risk prevention and control" and "data compliance audit" are extracted from financial security standards, and "teaching content accuracy" and "student information protection" are extracted from education standards. These keywords are like labels, clearly and accurately marking the security assessment standards of the corresponding technical fields, facilitating the subsequent rapid positioning and matching of related content, and improving the efficiency and accuracy of the entire assessment process. When generating security assessment dimensions for comprehensive evaluation of the target model, these security assessment keywords are combined with model feature information to generate dimensions through clustering algorithms, making the generated dimensions more scientific, reasonable and targeted. Clustering algorithms can aggregate similar concerns based on the degree of correlation between keywords and model features. For example, for an educational auxiliary language model, combining the keyword "accuracy of teaching content" and the application scenarios of the model, such as assisting teachers in preparing lessons and answering questions for students, can generate dimensions such as "evaluation of accuracy of teaching knowledge output". In this way, the generated security assessment dimensions are closely centered on the characteristics of the target model itself, and fully integrate the key points in the multi-field security assessment standards. They can fully and deeply dig out the potential security risks of the model. Whether it is content, data processing, or stability in the face of special situations, they can all be effectively evaluated, laying a solid foundation for ensuring the safe and reliable application of the target model in the corresponding field, and minimizing the impact of security risks on actual business development.

[0066] Exemplarily, in the embodiments of the present application, a dynamic clustering method of semantic similarity may be further introduced to implement the above steps.

[0067] First, semantic vectorization is performed on the model feature information and security assessment keywords. A pre-trained word vector model can be used to convert textual information and keywords into vector representations. For example, for the "psychological counseling model" in the model features and the security assessment keyword "patient privacy protection", the corresponding semantic vectors are obtained respectively. Then, the semantic similarity between these vectors is calculated. For example, a measurement method such as cosine similarity is used to determine the degree of similarity between the model features and each security assessment keyword. In the clustering process, clusters are divided dynamically based on semantic similarity instead of simply based on fixed rules. If the semantic similarity between "psychological counseling model" and "patient privacy protection" is very high, and there is also a high correlation with "data encryption technology" (calculated through semantic vectors), then they can be clustered into a "psychological counseling model privacy protection and encryption assessment" dimension.

[0068] This dynamic clustering can also be adjusted in real time based on model updates or new security requirements. For example, when new security technologies or regulations cause the keyword "data anonymization" to become more important in the psychological field, the clustering algorithm can automatically incorporate it into the relevant security assessment dimension by recalculating semantic similarity.

[0069] In addition, this dynamic clustering can also adapt more flexibly to new security concepts and changes in model characteristics. Because it is a dynamic clustering based on semantic similarity, it can promptly capture the slight changes in new security terms or model application scenarios in the industry, and generate more timely and targeted security assessment dimensions. For example, in the field of network security, new attack methods are constantly emerging, and semantic similarity dynamic clustering can quickly combine relevant model characteristics and new security assessment keywords into new assessment dimensions, such as "model robustness assessment dimensions for responding to new phishing attacks."

[0070] Alternatively, in another example, hierarchical clustering can be integrated with domain knowledge to assist in the adaptive generation of security assessment dimensions.

[0071] First, a hierarchical clustering method is used to divide the model feature information and security assessment keywords into coarse hierarchical categories. For example, from a macro perspective, the model features are divided into categories such as "data processing related", "content generation related", and "user interaction related", and the security assessment keywords are also divided into similar large categories, such as "data security keywords", "content security keywords", and "interaction security keywords". Then, in each hierarchical category, the clustering is further subdivided in combination with the domain knowledge graph. Taking the model features and "data security keywords related to "data processing" as an example, using the knowledge graph in the financial field, the model features involving financial transaction data and keywords such as "fund transfer security" and "account information protection" are clustered into "financial transaction data processing security assessment dimensions"; for the psychological field, the model features involving patient medical record data and keywords such as "medical record data privacy" and "psychological data compliance storage" are clustered into "psychological medical record data processing security assessment dimensions". This hierarchical clustering can be flexibly expanded according to the depth and breadth of domain knowledge. For example, as the security requirements for psychological data increase, under the "psychological medical record data processing security assessment dimension", hierarchical clustering can be further subdivided into more detailed content such as the "electronic medical record signature security assessment sub-dimension".

[0072] It not only takes into account the macro-model functions and security domain divisions, but also deeply combines specific industry knowledge to generate detailed security assessment dimensions. This helps to establish a common assessment framework between different fields, and can also be deeply customized according to the special requirements of each field. For example, there are data processing needs in both education and finance, but through the fusion of hierarchical clustering and domain knowledge, it is possible to generate "security assessment dimensions for personalized learning recommendations using education data" and "security assessment dimensions for risk assessment using financial data", respectively, making the security assessment dimensions more in line with actual application scenarios.

[0073] In addition to model feature information and security assessment keywords in text form, other modal data are also considered. For example, if there is a visual architecture diagram of the model, a data flow diagram, etc., these multimodal information are converted into a form that can be clustered together with the text information. For the visual architecture diagram, the key structural features can be extracted through image recognition technology, and converted into vector representation, and clustered together with the text vector.

[0074] In the clustering process, user feedback is introduced as a basis for adjusting clustering. For example, security assessors can annotate or evaluate the preliminary clustering results, such as believing that the security assessment dimension in a certain cluster is not comprehensive enough or too broad. The clustering algorithm can adjust the clustering strategy based on these feedbacks using technologies such as reinforcement learning. If users feedback that a cluster related to a content review model lacks consideration of emerging social media content formats, the clustering algorithm can reintegrate model features (such as the model's ability to handle new formats of content) and security assessment keywords (such as "detection of harmful information in new social media format content") to generate a more complete security assessment dimension.

[0075] Multimodal clustering can make full use of various information sources of the model and avoid information loss caused by relying solely on text information. Combined with user feedback, the clustering algorithm has the ability to self-learn and optimize, and can continuously improve the quality and practicality of the generated security assessment dimensions. This method can generate dimensions that are more in line with actual assessment needs when faced with complex and changing models and diverse security requirements. For example, in the security assessment of multimedia content processing models, more accurate dimensions such as "multimedia content copyright protection assessment dimensions that combine video architecture and text information" can be generated.

[0076] In the embodiment of the present application, the abnormal use case in step 102 is used to simulate abnormal input data of the target model. Further, it is assumed that the abnormal use case includes at least one of the following: a test case containing unsafe input, a test case with potential privacy leakage, a test case containing bias prompt words, a test case containing abnormal data format, and a test case containing damaged data.

[0077] Based on the above assumptions, in an optional embodiment, the autonomous test case of generating the target model based on the security assessment dimension in 102 can be implemented as follows:

[0078] A corresponding small-scale language model is constructed based on the security assessment dimension; the parameter scale of the small-scale language model is smaller than the parameter scale of the target model; based on the generative adversarial network contained in the small-scale language model, an initial test case matching the security assessment dimension is generated; the initial test case contains abnormal data samples associated with the security assessment dimension; the initial test case is adaptively expanded and screened by an auxiliary expansion model to obtain candidate test cases; the candidate test cases are automatically annotated by an auxiliary annotation model to obtain the autonomous test case.

[0079] In the above embodiment, the small-scale language model is constructed in order to more efficiently and specifically generate test cases suitable for the target large language model. Since the parameter scale of the small-scale language model is smaller than that of the target model, training and operation are relatively simple and convenient. The small-scale language model can focus on the content and functions involved in a specific security assessment dimension, and simulate some of the behaviors of the target model in the corresponding security aspects, so as to subsequently generate test cases closely related to it. For example, if the target model is a large language model for financial services, and the security assessment dimension includes "psychological counseling risk prediction accuracy assessment", then the constructed small-scale language model can focus on psychological counseling data processing and simple risk prediction functions to build it so that it can reflect the characteristics related to this security dimension.

[0080] Determine the architecture and number of parameters of the small-scale language model based on the specific security assessment dimension. For example, choose a relatively simple neural network architecture and set the parameter scale to the million level (much smaller than the billions or tens of billions that the target model may have). If the target is the "patient privacy protection assessment" dimension of the large language model in the field of psychological counseling, the training data of the small-scale model can select some public and simplified psychological case data, focusing on learning how to process text content containing basic patient information, so as to simulate some situations of the target model in terms of patient privacy data processing.

[0081] Furthermore, the advantage of using a generative adversarial network, which consists of a generator and a discriminator, to generate initial test cases is that it can create diverse input data that fits a specific security assessment dimension. The generator can generate inputs that are likely to trigger security vulnerabilities in the target model based on the features and patterns learned by the small-scale language model, and in particular, it can generate content containing abnormal data samples. These abnormal data samples are closely related to the corresponding security assessment dimension, making it easier to detect the security of the target model in this dimension. For example, in the "harmful information detection and evaluation" dimension of the content review large language model, the GAN generator can generate text content containing abnormal data such as various illegal words and sensitive topics as initial test cases.

[0082] The GAN architecture is embedded in the constructed small-scale language model. For the generator part, it is trained to learn the potential data distribution pattern related to the security assessment dimension. Taking the dimension of "Product False Advertising Content Detection and Evaluation" of the large language model for e-commerce recommendation as an example, the generator will generate some product recommendation texts containing abnormal data samples such as exaggerating product efficacy and fabricating product reviews based on the learning of product description data by the small-scale model as initial test cases. The discriminator constantly competes with the generator to improve the quality and pertinence of the generated content, so that it better meets the needs of detecting security vulnerabilities of the target model under this security assessment dimension.

[0083] Next, although the initial test cases are already related to the security assessment dimension, they may still be insufficient in terms of quantity, diversity, etc. The role of the auxiliary expansion model is to further expand and optimize them. On the one hand, the number and coverage of test cases are increased through adaptive expansion, so that it can more comprehensively detect possible security issues of the target model under this security dimension; on the other hand, it is screened to remove duplicate, redundant or non-compliant content, improve the quality and effectiveness of test cases, and finally obtain more representative and practical candidate test cases.

[0084] A variety of technical means can be used to assist in the expansion of the model. For example, random sampling, combination, and structural variation can be used for expansion. Taking the "news authenticity assessment" dimension of the news generation large language model as an example, for some text content about false news in the initial test case, random sampling can select false content of similar styles from different angles; combination technology can combine different false event descriptions, characters and other elements to create new false news test cases; structural variation technology can change the text's expression method, grammatical structure, etc. to generate more variants. Then, through the semantic similarity algorithm and other methods, it is screened to remove those contents with too high semantic repetition, leaving diverse and effective candidate test cases, such as false news test cases constructed from different event subjects, plots, etc. after screening.

[0085] Finally, automatic labeling can clearly identify the security assessment dimensions and expected risk levels corresponding to each test case, making it easier to use these autonomous test cases to quickly and accurately analyze the performance and risk level of the target model in each dimension when performing security testing on the target model. The auxiliary labeling model comprehensively and accurately labels candidate test cases based on certain rules and algorithms, ensuring that the management and use of test cases are more orderly and efficient.

[0086] The annotation rules are pre-set in the auxiliary annotation model. For example, for a candidate test case under the dimension of "inappropriate speech content detection and evaluation" of a large social chat language model, if one of the test cases is a chat sentence containing insulting words, the annotation model will annotate it as the corresponding "inappropriate speech content detection and evaluation" dimension according to the rules, and at the same time, based on the severity of the insulting words, the possible social impact, and other factors, the expected risk level will be annotated as high risk or medium risk, etc. For candidate test cases in different fields and different security assessment dimensions, the annotation model can accurately complete the annotation work according to the corresponding rules, and finally form a complete set of autonomous test cases with clear annotation information, which will be used for subsequent security testing of the target model.

[0087] As an optional embodiment, in the above steps, the initial test cases are adaptively expanded and screened by the auxiliary expansion model to obtain candidate test cases, including:

[0088] A plurality of initial test cases are randomly selected, and the selected initial test cases are combined to obtain a first candidate test case; based on the test scenario structure in the application scenario, corresponding initial test cases are selected and combined into a second candidate test case; the initial test cases are structurally deformed and combined into a third candidate test case; a semantic similarity algorithm is used to screen candidate test cases whose similarity is lower than a set similarity threshold from the first candidate test case, the second candidate test case, and the third test case as the candidate test cases for final output.

[0089] Specifically, first of all, by randomly selecting and combining initial test cases, we aim to increase the diversity and complexity of test cases, so as to more comprehensively cover various situations that the target model may encounter under specific security assessment dimensions. Different initial test cases themselves contain different abnormal data samples or specific input patterns. Combining them together can create new and more challenging test scenarios, which helps to uncover potential security vulnerabilities in the model. For example, a single initial test case may only trigger a relatively simple abnormal reaction of the model, but the combined test case may trigger complex security issues of the model under the interaction of multiple factors.

[0090] For example, suppose you are generating test cases for a large language model for content review, and its security assessment dimension is "harmful information detection assessment". The initial test cases contain text content with different banned words (such as insulting words, sensitive political words, etc.) and some sentences containing false information. Randomly select several of the initial test cases, such as a sentence with insulting words and a sentence containing false news content, and combine them into a new text paragraph. This new combination becomes the first candidate test case. Through a large number of such random selection and combination operations, many first candidate test cases of different forms and contents can be generated, and the model's ability to identify complex harmful information can be tested from more angles.

[0091] Furthermore, considering the test scenario structure in the application scenario to select and combine the initial test cases is to make the generated test cases more in line with the environment and business logic of the actual use of the target model. Different application scenarios have their own unique processes, rules, and data association methods. Building test cases according to these characteristics can simulate more realistic and targeted security risk scenarios, making the security detection of the model more in line with the actual situation and improving the effectiveness and reference value of the detection results.

[0092] Taking the e-commerce recommendation large language model as an example, its application scenarios involve multiple links such as users browsing products, viewing details, and adding to shopping carts. The corresponding test scenario structure also covers the input situations that may appear in these different stages. If the security assessment dimension is "product recommendation accuracy and compliance assessment", the initial test case includes inaccurate text descriptions of products, slogans that exaggerate the efficacy of products, etc. Then according to the test scenario structure, when users view product details, select the initial test case containing exaggerated descriptions of product efficacy, and then combine it with the initial test case of false propaganda of product discount information that may appear when users add to shopping carts. Combine them to form a second candidate test case that conforms to the logic of the e-commerce recommendation process, so as to detect the model's ability to handle different types of illegal recommendation information in the entire recommendation business chain.

[0093] Then, the initial test cases are structurally deformed and combined to further expand the test case variations and avoid test cases being too limited and single. By changing the grammatical structure of the text, the order of data arrangement, the nesting of elements and other structural changes, new test cases that appear different but are essentially still around the same security assessment dimension can be created, which increases the richness of the test cases and helps to more carefully detect the model's response to different forms of input and discover potential security risks that may be exposed due to changes in the input structure.

[0094] Still taking the "harmful information detection and evaluation" dimension of the content review large language model as an example, for sentences containing false information in the initial test cases, such as "a well-known brand has released a magical drug that can cure all diseases", the structure can be deformed to "it is said that a magical drug has been released by a well-known brand, and this drug claims to cure all diseases". Then, multiple initial test cases that have undergone such structural deformations are combined to form a third candidate test case. Such changes may affect the model's ability to parse semantics and identify harmful information. The test cases generated in this way can more comprehensively examine the model's detection effect when faced with harmful information in various forms of expression.

[0095] Finally, after a large number of candidate test cases are obtained through the above methods, some semantically similar content will inevitably appear. If these similar test cases are all used to detect the target model, it will not only increase the detection cost and time, but also may lead to repeated testing of certain security risks, reducing detection efficiency. Through the semantic similarity algorithm, the semantic similarity between test cases can be measured, and test cases with similarity below the set threshold can be screened out. Candidate test cases with obvious differences that can detect model security from different angles are retained, making the final test case set more refined, efficient and comprehensive.

[0096] For example, assuming that the similarity threshold is set to 0.8 (the specific value can be adjusted according to actual conditions and needs), the first, second, and third candidate test cases are compared pairwise using a semantic similarity algorithm (such as a cosine similarity algorithm, etc.). For example, for two first candidate test cases, one is "This product can definitely make you ten years younger instantly, and there are no side effects. Come and buy it", and the other is "Using this product will make you ten years younger immediately, without any side effects, hurry up and buy it". After the semantic similarity algorithm calculates their similarity as 0.9, which is higher than the set threshold, one of them will be screened out. For other candidate test cases with large semantic differences, such as "This product contains harmful ingredients, everyone should never buy it", the similarity is lower than that of the previous test case, and it will be retained as the final output candidate test case for subsequent security testing of the target model.

[0097] As an optional embodiment, in the above steps, it is assumed that the label of the autonomous test case is used to indicate the corresponding security assessment dimension and expected risk level. Based on this, the candidate test case is automatically labeled by the auxiliary labeling model to obtain the autonomous test case, including:

[0098] Identify the core security risks involved in the candidate test case; annotate at least one security assessment dimension corresponding to the core security risk into the candidate test case as a security assessment dimension label of the candidate test case; and predict and annotate the expected risk level of the candidate test case based on the risk level classification method corresponding to at least one security assessment dimension.

[0099] Among them, the risk level classification method includes at least: the number of risk levels, the range of risk levels, and the weight of risk factors.

[0100] For example, in a psychological counseling scenario, assume that there is a series of candidate test cases that aim to perform security testing on a large language model used for psychological counseling services.

[0101] First, identify the core security risks involved in the candidate test cases. For example, some candidate test cases are dialogue inputs that intentionally induce the consultant to have extremely negative emotions, so the core security risk involved is that it may cause serious harm to the consultant's mental health. Some candidate test cases contain statements that may leak the consultant's privacy, so the core security risk is the problem of privacy leakage.

[0102] Next, for these core security risks, the corresponding security assessment dimensions are annotated to the candidate test cases as security assessment dimension labels. For example, for test cases that induce extreme negative emotions, the security assessment dimension label "Psychological Counseling Suggestion Security Assessment" is annotated, because it is related to whether the psychological guidance given by the model is appropriate and safe; for test cases involving privacy leakage risks, the dimension label "Consultant Privacy Protection Assessment" is annotated to make it clear that this test case is mainly aimed at testing the model in protecting the privacy of consultants.

[0103] Then, the expected risk level of the candidate test cases is predicted and marked based on the risk level classification method corresponding to each security assessment dimension. Taking the dimension of "Psychological Counseling Suggestion Security Assessment" as an example, if the number of risk levels is divided into three levels: high, medium, and low, when considering the weight of risk factors, the degree of psychological harm to the consultant and the scope of impact are taken as important weight factors. If the test case that induces extreme negative emotions is likely to cause the consultant to have a serious psychological crisis and the scope of impact is also wide, according to this risk level classification method, its expected risk level will be marked as high risk. Looking at the dimension of "Consultant Privacy Protection Assessment", if the risk level range is determined according to the sensitivity of the privacy information that may be leaked, if the test case contains extremely private personal experiences of the consultant and the consequences of leakage are very serious, it will also be marked as a high risk level after comprehensive judgment based on the corresponding risk factor weights.

[0104] Through this automatic labeling process, each candidate test case has a clear security assessment dimension label and expected risk level label, which together constitute an autonomous test case. This allows us to clearly know the security aspects that each test case focuses on testing and its potential risk level when using these autonomous test cases to conduct security testing on the psychological counseling large language model in the future, which facilitates a comprehensive and targeted analysis of the model's security status, and accurately discovers potential security risks that may exist in the model in psychological counseling services, such as giving inappropriate advice and poor privacy protection, thereby providing a strong basis for the optimization and improvement of the model and ensuring the security of psychological counseling services and the rights and interests of counselors.

[0105] It is understandable that in actual applications, it is necessary to design corresponding induction modes based on the application scenarios of different models and the security dimensions expected to be detected. For example, for an intelligent customer service chatbot model, if you want to test its security in handling sensitive topics, you can input sentences that are highly inductive and involve sensitive information, such as "I heard that your company has serious tax evasion problems. Is that true?" to see whether the model's response is appropriate, whether it will leak internal information that should not be disclosed, or give inappropriate confirmation responses.

[0106] Taking the content recommendation large language model as an example, if you want to see whether it is easy to be induced to recommend bad content, you can input "I am very interested in some exciting and horror niche movies recently, please recommend a few to me". If the model recommends movies that contain a lot of violence and horror and do not meet the content review standards, it means that the model has security loopholes in the compliance of content recommendation and can be easily triggered by specific induction patterns to generate bad outputs.

[0107] As an optional embodiment, in 103, the autonomous test case is input into the target model to execute the model function, and the response result of the target model to the autonomous test case is obtained, including:

[0108] A preset pattern induction strategy is adopted to generate model input data matching the autonomous test case through an auxiliary induction model; the model input data includes abnormal input data; the abnormal input data includes at least one of the following: fault data, privacy data, bias prompt words, abnormal data format, damaged data; the model input data is input into the target model to execute the model function, and the response result of the target model to the model input data is obtained.

[0109] Furthermore, in 104, a security assessment report for the target model is generated based on the response result, including: building a monitoring strategy through a dynamic monitoring model based on the security assessment dimension, and dynamically monitoring the real-time operating status of the target model based on the monitoring strategy; using a security assessment model to perform a multi-level security assessment based on the response result and the real-time operating status to obtain the security assessment report; wherein the multi-level security assessment includes at least: content security assessment, privacy protection assessment, and model robustness assessment of the target model.

[0110] For example, when conducting security assessments on large language models, the above steps have important and multifaceted effects. First, when inputting autonomous test cases into the target model to obtain response results, it is critical to use a pre-set pattern induction strategy and use an auxiliary induction model to generate model input data containing various types of abnormal input data. For example, for large language models, the input of fault data can simulate data errors that may occur in actual applications, such as garbled characters and semantically incoherent content in text input, or damaged and unclear image data input in image recognition large language models, so as to detect whether the model can handle such anomalies correctly, or whether it will crash, output wrong results, and other security-affecting situations. As abnormal input data, privacy data can be used to verify whether the large language model will improperly process, store, or leak content involving privacy information when receiving it. After all, many large language model application scenarios involve a large amount of user privacy, which is crucial to protecting user rights. The introduction of bias prompt words can examine whether the large language model will be induced to output content that is biased, discriminatory, and inconsistent with public order and good customs in terms of content generation. For example, in functional applications such as text creation and intelligent question and answer, if the model is easily guided by bias prompt words to generate bad content, it may cause adverse social impacts in actual use. The input of abnormal data formats targets the compatibility and processing capabilities of the large language model for data in different formats, avoiding security vulnerabilities caused by format problems, such as the response performance of the model when the input does not meet the requirements of file formats, data structures, etc. Damaged data can also test the stability and accuracy of the model under complex conditions such as incomplete data and partial information loss. Through such diverse abnormal input data, the safety hazards of the large language model in various boundary conditions and potential dangerous scenarios can be fully explored, and the response results obtained truly reflect the performance of the model under different stress tests.

[0111] Furthermore, from the perspective of generating a security assessment report based on the response results, it is of great significance to construct a monitoring strategy based on the security assessment dimension through a dynamic monitoring model, and dynamically monitor the real-time operation status of the target model based on this. During the operation of the large language model, its internal state is complex and changeable. Through dynamic monitoring, the operation of the model under different stages and different inputs can be captured in real time, such as observing its resource usage, changes in processing time, output stability and other indicators when processing a large amount of text input. This helps to understand the overall performance and potential security risks of the model more comprehensively and deeply. The use of a security assessment model for multi-level security assessment, covering content security assessment, privacy protection assessment, model robustness assessment and other aspects, is more in line with the characteristics and application requirements of the large language model. Content security assessment can detect whether the content generated by the large language model is compliant, whether there is false information, whether there is any problem of inducing bad behavior, etc., to ensure that the output text, images and other types of content will not cause harm in different application scenarios. Privacy protection assessment is extremely important for large language models that contact and process a large amount of user data. It can accurately determine whether the model has a risk of privacy leakage and whether the protection measures for privacy data are in place. The model robustness assessment focuses on the ability of the large language model to resist various abnormal inputs, malicious attacks, etc., such as whether the model can maintain stable output in the face of attacks, whether it can correctly identify and reject abnormal data formats, etc. Through such a comprehensive and multi-level assessment, the security assessment report finally generated can accurately and detailedly present the security status of the large language model, and provide strong data support for the subsequent optimization, adjustment, reasonable deployment and security policy formulation of the model, ensuring the security, reliability and compliance of the large language model in actual applications, so that it can better serve many business scenarios, while avoiding various adverse consequences caused by security issues, such as user information leakage, the spread of bad information affecting social stability, etc.

[0112] As an optional embodiment, the method of adopting a preset pattern induction strategy to generate model input data matching the autonomous test case through an auxiliary induction model includes:

[0113] A fault injection strategy matching the autonomous test case is obtained from the pattern induction strategy; wherein multiple fault injection strategies are respectively associated with different data types, different technical fields, and different data contents; based on the matching fault injection strategy, the autonomous test case is processed by the auxiliary induction model to obtain the model input data. The data processing includes but is not limited to one of the following: clipping, insertion, random arrangement, content logic modification, and value modification.

[0114] Specifically, first, the fault injection strategy that matches the autonomous test case is obtained from the pattern induction strategy. Since these fault injection strategies are respectively associated with different data types (such as text, images, numerical data types), different technical fields (such as medical, financial, education and other fields) and different data contents, the generated model input data is highly targeted. For example, for a large language model in the medical field, when processing autonomous test cases containing text data such as patient medical records, the corresponding fault injection strategy can be used to accurately simulate data problems that may occur in actual medical application scenarios, such as trimming medical record text, simulating the loss of some information during data transmission, or inserting some erroneous diagnostic descriptions to test the model's ability to recognize abnormal information.

[0115] Next, the auxiliary induction model is used to perform corresponding data processing on the autonomous test cases according to the matching fault injection strategy. Whether it is a trimming operation, which can change the integrity of the data and test the model's processing ability when facing incomplete data; or an insertion operation, which can artificially create "impurities" in the data to observe whether the model will be disturbed by it and output incorrect results; random arrangement can disrupt the original logical order of the data and test the model's understanding and reconstruction ability of semantics and logical relationships. For example, a medical record text that records the development of the disease in chronological order is randomly arranged and input into the model to see if it can correctly analyze the disease; content logic modification can deliberately create contradictory and unreasonable content to see if the model can identify these unreasonable data and avoid being misled in actual applications; numerical modification is for numerical data, changing its size, range, etc., simulating data entry errors, etc., such as modifying the transaction amount in the large language model in the financial field to test the model's handling of abnormal values ​​and the accuracy of transaction risk judgment.

[0116] In general, by generating model input data through such a series of data processing, we can comprehensively and effectively simulate the data situations that the large language model may receive in various complex and abnormal actual usage scenarios, so as to fully explore the potential security vulnerabilities of the model, and allow the subsequent security evaluation report generated based on the model's response to these input data to more truly and accurately reflect the model's security performance, providing a solid foundation for ensuring the safe and reliable application of large language models in various fields, and avoiding security issues such as outputting wrong suggestions, leaking privacy, and system crashes in actual use due to failure to take these abnormal data situations into consideration.

[0117] It is understandable that in actual applications, different fault injection methods are designed according to the data types and characteristics processed by the model. For models that process text data, a complete text can be truncated to make it incomplete, or some garbled characters, incorrect punctuation marks, etc. can be deliberately inserted into the text to change the normal logic and format of the text before entering the model. For example, in the test of a document automatic summary generation model, a long text with incoherent sentences, confusing logic and garbled characters is input to see whether the model can process it normally and generate reasonable summary content. If the model crashes or the generated summary is completely unreasonable, it means that the model's ability to handle such text data failures is insufficient and there are security vulnerabilities.

[0118] Taking the financial data analysis big language model as an example, we can deliberately input some data that does not conform to the logic of financial data, such as setting an unreasonable timestamp order in time series data, or setting the price of a certain stock to a maximum or minimum value beyond the normal range, and observe the model's analysis results of these data. If the model gives obviously unreasonable investment advice based on these erroneous data or makes calculation errors, it means that the model cannot guarantee safe and reliable output when facing such data failures, and there are corresponding security risks.

[0119] As an optional embodiment, in 104, after generating a security assessment report for the target model based on the response result, the security vulnerabilities and potential risks contained in the security assessment report can also be identified. Then, an optimization strategy for security vulnerabilities and potential risks, and an optimization evaluation index matching the optimization strategy are generated through an auxiliary optimization model. The optimization strategy includes at least: model parameter fine-tuning, training data adjustment, and incremental training. Finally, the target model is optimized based on the optimization strategy, and the optimized target model is security evaluated using the optimization evaluation index to update the security assessment report of the target model.

[0120] In the security assessment process based on large language models, after accurately identifying security vulnerabilities and potential risks from the security assessment report, we can further determine the security risks to further improve the security of model operation. For example, if it is found that the model will mistakenly generate content that violates security regulations when facing a specific type of malicious input, this is an obvious security vulnerability. Or when the model responds too long and the accuracy rate decreases under high load operation, this is a manifestation of potential risks.

[0121] Using auxiliary optimization models to generate optimization strategies is a key step. Model parameter fine-tuning is a delicate operation that improves the performance of the model by making targeted, small adjustments to the massive parameters within the model. For example, if it is found that the model performs poorly on a certain type of semantic understanding task, the weights of the relevant neural network connections can be adjusted to enable it to capture semantic information more accurately, thereby reducing security issues caused by semantic understanding deviations.

[0122] Adjusting the training data is to solve the problem from the source of the data. If it is determined that the security vulnerability of the model is related to the quality or distribution of the training data, such as too much noise data, missing data of a specific type, or inaccurate data labeling, the training data can be cleaned, supplemented, or re-labeled. For example, if it is found that the model makes mistakes when processing text of a certain type of rare event, it may be because the number of samples of such events in the training data is too small. At this time, the relevant sample data can be added to allow the model to better learn the characteristics of such events, thereby improving the security of dealing with such situations.

[0123] Incremental training enables the model to continuously learn new knowledge and adapt to new situations based on existing knowledge and capabilities. With the development of technology and changes in application scenarios, new security requirements, data types, and language expressions continue to emerge. Through incremental training, the model can learn new vocabulary semantics, new security rules, and new domain knowledge, so as to better cope with increasingly complex and changing security challenges. For example, when some new buzzwords or new terms for a specific industry appear on the Internet, incremental training allows the model to quickly understand and correctly process texts containing these words, avoiding security risks caused by outdated knowledge.

[0124] After optimizing the target model based on these optimization strategies, the model is again evaluated for security using the matching optimization evaluation indicators. These optimization evaluation indicators are specifically set according to the goals and characteristics of the optimization strategy, and can accurately measure the improvement in the model's security performance after optimization. For example, the evaluation indicators after fine-tuning the model parameters may focus on whether the model's accuracy and recall rate on specific security test cases have been improved; for the evaluation after the training data is adjusted, the model's processing effect on the data of the previous weak links and the stability of the overall data processing will be examined; and after incremental training, the model's adaptability to new knowledge and scenarios and whether new security vulnerabilities have been generated will be tested. Through such an evaluation process, the security status of the model after optimization can be fully and accurately understood, and the security evaluation report of the target model can be updated accordingly, so that the report can truly reflect the latest security level of the model, provide a reliable basis for the model's subsequent application, further optimization, and security management, and form a closed-loop process for continuous improvement and ensuring the safe operation of the model.

[0125] As an optional embodiment, after generating a security assessment report for the target model based on the response result in 104, the security assessment knowledge graph corresponding to the technical field may also be updated based on the security assessment dimension, the autonomous test case and the security assessment report.

[0126] After completing the generation of the security assessment report for the target model, it is a very valuable follow-up work to update the security assessment knowledge graph corresponding to the technical field based on the security assessment dimensions, autonomous test cases and security assessment reports. For a specific technical field, the security assessment knowledge graph originally brings together many knowledge elements related to the security assessment of large language models in this field, such as common types of security risks, past security vulnerability cases, and effective countermeasures. During the security assessment process for the target model this time, the security assessment dimensions determined comprehensively cover the security requirements that the model should meet in all aspects. This may include some newly emerging dimensions that have not been highlighted in the knowledge graph before, such as new privacy risk dimensions that have emerged with the application of new technologies. Integrating these new dimensions into the knowledge graph can make its coverage of security assessment more comprehensive.

[0127] Autonomous test cases should not be ignored either. These cases are carefully designed to detect the target model. The various normal and abnormal situations involved, such as boundary value test cases for specific functional inputs and cases containing new abnormal data, reflect the specific practice of exploring model security in actual evaluation. Adding information such as typical scenarios, input modes, and corresponding expected detection effects represented by these autonomous test cases into the knowledge graph can enrich its content and provide more reference examples for subsequent security evaluations of similar models. The security evaluation report is even more important. It presents in detail the actual performance of the target model under various security assessment dimensions, including discovered security vulnerabilities, potential risks, and overall security status. These results based on real evaluations are fed back to the knowledge graph, which can update existing knowledge on security risk judgment, severity classification, etc. For example, if a new attack method is found that can easily break through the model protection, the corresponding response strategy can be added to the knowledge graph and the relevant risk level description can be updated.

[0128] Through this updating process, the security assessment knowledge graph corresponding to the technical field can be continuously improved, which can better meet the ever-changing security assessment needs of large language models in practical applications, and provide more accurate, comprehensive and up-to-date knowledge support for subsequent model security assessments in the same field or similar scenarios, so that security assessment work can more efficiently and accurately discover potential security risks of the model, and ensure the safe and reliable application of large language models in the corresponding technical fields.

[0129] In the technical solution of this application, by constructing security assessment dimensions and generating adaptive autonomous test cases to actively evaluate the security of the model, an automated and comprehensive evaluation of the model is achieved, the efficiency and accuracy of the model security assessment is improved, the adaptability to models of different fields and scales is enhanced, the automation and continuous optimization of security assessment is achieved, and the security of model operation is ensured.

[0130] In another embodiment of the present application, a security assessment device based on a large language model is also provided. Figure 2 The device comprises the following units:

[0131] The identification unit is configured to identify, for a target model to be evaluated, a security assessment dimension matched by the target model; the target model is a large language model;

[0132] A generating unit is configured to generate an autonomous test case of a target model based on the security assessment dimension; the autonomous test case includes a normal user and an abnormal case; the information content in the abnormal case corresponds to the type of the target model;

[0133] An evaluation unit is configured to input the autonomous test case into the target model to execute the model function and obtain a response result of the target model to the autonomous test case;

[0134] A reporting unit is configured to generate a security assessment report for the target model based on the response result.

[0135] Further optionally, the identification unit, for the target model to be evaluated, identifies the security assessment dimension matched by the target model, and is configured to:

[0136] Performing model characteristic analysis on the target model to obtain model characteristic information of the target model; the model characteristic information at least includes: application scenario, model structure, and training data of the target model; the application scenario at least includes: model function of the target model and the technical field to which the model belongs;

[0137] Generate a security assessment dimension for comprehensively evaluating the target model based on the model characteristic information and the preset security assessment criteria;

[0138] An expert system is used to verify and optimize the generated security assessment dimensions to obtain the final output security assessment dimensions; the expert system is obtained by adaptively configuring the security assessment knowledge graph based on the technical field to which the model belongs.

[0139] Further optionally, the identification unit is further configured to: for the target model to be evaluated, before identifying the security assessment dimension matched by the target model, obtain security assessment standards in multiple technical fields;

[0140] Perform keyword extraction processing on the security assessment standard to obtain security assessment keywords corresponding to multiple technical fields; wherein the keyword extraction processing at least includes: semantic recognition and semantic retrieval;

[0141] Using the safety assessment keywords, marking the safety assessment standards corresponding to multiple technical fields;

[0142] The generating, based on the model characteristic information and the preset security assessment criteria, a security assessment dimension for comprehensively assessing the target model comprises:

[0143] Based on the model characteristic information and the security assessment keywords, a security assessment dimension for comprehensively evaluating the target model is generated through a clustering algorithm.

[0144] Further optionally, the abnormal use case is used to simulate abnormal input data of the target model; the abnormal use case includes at least one of the following: a test case containing unsafe input, a test case with potential privacy leakage risks, a test case containing bias prompt words, a test case containing abnormal data format, and a test case containing damaged data.

[0145] Further optionally, the generating unit generates an autonomous test case of the target model based on the security assessment dimension, and is configured to:

[0146] Building a corresponding small-scale language model based on the security assessment dimension; the parameter scale of the small-scale language model is smaller than the parameter scale of the target model;

[0147] Based on the generative adversarial network included in the small-scale language model, an initial test case matching the security assessment dimension is generated; the initial test case includes an abnormal data sample associated with the security assessment dimension;

[0148] Adaptively expanding and screening the initial test cases through the auxiliary expansion model to obtain candidate test cases;

[0149] The candidate test cases are automatically annotated by an auxiliary annotation model to obtain the autonomous test cases.

[0150] Further optionally, the generating unit adaptively expands and screens the initial test cases through the auxiliary expansion model to obtain candidate test cases, and is configured to:

[0151] Randomly selecting multiple initial test cases, and combining the selected multiple initial test cases to obtain a first candidate test case;

[0152] Based on the test scenario structure in the application scenario, select corresponding initial test cases and combine them into second candidate test cases;

[0153] Perform structural deformation on the initial test cases and combine them into third candidate test cases;

[0154] A semantic similarity algorithm is used to screen candidate test cases whose similarity is lower than a set similarity threshold from the first candidate test case, the second candidate test case, and the third test case as candidate test cases for final output.

[0155] Further optionally, the label of the autonomous test case is used to indicate the corresponding security assessment dimension and expected risk level; the generating unit automatically labels the candidate test case through the auxiliary labeling model to obtain the autonomous test case, and is configured to:

[0156] Identify the core security risks involved in the candidate test cases;

[0157] Labeling at least one security assessment dimension corresponding to the core security risk to the candidate test case as a security assessment dimension label of the candidate test case;

[0158] Based on the risk level classification method corresponding to at least one security assessment dimension, the expected risk level of the candidate test case is predicted and labeled; wherein the risk level classification method at least includes: the number of risk levels, the range of risk levels, and the weight of risk factors.

[0159] Further optionally, the evaluation unit inputs the autonomous test case into the target model to execute the model function and obtains the response result of the target model to the autonomous test case, and is configured as follows:

[0160] A preset pattern induction strategy is adopted to generate model input data matching the autonomous test case through an auxiliary induction model; the model input data includes abnormal input data; the abnormal input data includes at least one of the following: fault data, privacy data, bias prompt words, abnormal data format, and damaged data;

[0161] The model input data is input into the target model to execute the model function, and the response result of the target model to the model input data is obtained.

[0162] The reporting unit generates a security assessment report for the target model based on the response result, and is configured to:

[0163] Based on the security assessment dimension, a monitoring strategy is constructed through a dynamic monitoring model, and based on the monitoring strategy, a real-time operating status of the target model is dynamically monitored;

[0164] A security assessment model is used to perform a multi-level security assessment based on the response result and the real-time operating status to obtain the security assessment report; wherein the multi-level security assessment includes at least: content security assessment of the target model, privacy protection assessment, and model robustness assessment.

[0165] Further optionally, the evaluation unit adopts a preset pattern induction strategy to generate model input data matching the autonomous test case through an auxiliary induction model, and is configured as follows:

[0166] Obtaining a fault injection strategy matching the autonomous test case from the pattern induction strategy; wherein the plurality of fault injection strategies are respectively associated with different data types, different technical fields, and different data contents;

[0167] Based on the matching fault injection strategy, the autonomous test case is processed by the auxiliary induction model to obtain the model input data; wherein the data processing includes one of the following: trimming, inserting, shuffling, content logic modification, and numerical modification.

[0168] Further optionally, after the evaluation unit generates a security evaluation report for the target model based on the response result, it is further configured to:

[0169] Identify security vulnerabilities and potential risks contained in the security assessment report;

[0170] Generate optimization strategies for security vulnerabilities and potential risks and optimization evaluation indicators matched by the optimization strategies through auxiliary optimization models; the optimization strategies at least include: model parameter fine-tuning, training data adjustment, and incremental training;

[0171] The target model is optimized based on the optimization strategy, and the optimized target model is security evaluated using the optimization evaluation index to update the security evaluation report of the target model.

[0172] Further optionally, after the evaluation unit generates a security evaluation report for the target model based on the response result, it is further configured to:

[0173] Based on the security assessment dimensions, the autonomous test cases and the security assessment report, the security assessment knowledge graph corresponding to the technical field to which the target model belongs is updated.

[0174] The system can implement various steps in the above method embodiment, which will not be expanded here.

[0175] In the embodiment of the present application, a security assessment device based on a large language model is adopted to actively evaluate the security of the model by constructing security assessment dimensions and generating adaptive autonomous test cases, thereby realizing automated and comprehensive evaluation of the model, improving the efficiency and accuracy of model security assessment, enhancing the adaptability to models of different fields and scales, realizing automation and continuous optimization of security assessment, and ensuring the security of model operation.

[0176] See also Figure 3 , Figure 3 This is a schematic diagram of an electronic device provided in an embodiment of the present application. Figure 3 As shown, an embodiment of the present application provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 511, the following steps are implemented: for a target model to be evaluated, a security assessment dimension matched by the target model is identified; the target model is a large language model; an autonomous test case of the target model is generated based on the security assessment dimension; the autonomous test case includes normal users and abnormal cases; the information content in the abnormal case corresponds to the type of the target model; the autonomous test case is input into the target model to execute the model function, and a response result of the target model to the autonomous test case is obtained; and a security assessment report for the target model is generated based on the response result.

[0177] See also Figure 4 , Figure 4 A schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present application. Figure 4 As shown, this embodiment provides a computer-readable storage medium 600, on which a computer program 611 is stored. When the computer program 611 is executed by a processor, the following steps are implemented: for a target model to be evaluated, a security assessment dimension matched by the target model is identified; the target model is a large language model; an autonomous test case of the target model is generated based on the security assessment dimension; the autonomous test case includes normal users and abnormal cases; the information content in the abnormal case corresponds to the type of the target model; the autonomous test case is input into the target model to execute the model function, and a response result of the target model to the autonomous test case is obtained; and a security evaluation report for the target model is generated based on the response result.

[0178] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0179] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0180] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A security assessment method based on a large language model, characterized in that: The method comprises: For the target model to be evaluated, identify the security assessment dimension matched by the target model; the target model is a large language model; Generate autonomous test cases of the target model based on the security assessment dimension; the autonomous test cases include normal users and abnormal cases; the information content in the abnormal cases corresponds to the type of the target model; Inputting the autonomous test case into the target model to execute the model function, and obtaining the response result of the target model to the autonomous test case; Generate a security assessment report for the target model based on the response result; Wherein, for the target model to be evaluated, identifying the security assessment dimension matched by the target model includes: Performing model characteristic analysis on the target model to obtain model characteristic information of the target model; the model characteristic information at least includes: application scenario, model structure, and training data of the target model; the application scenario at least includes: model function of the target model and the technical field to which the model belongs; Generate a security assessment dimension for comprehensively evaluating the target model based on the model characteristic information and the pre-set security assessment criteria; An expert system is used to verify and optimize the generated security assessment dimensions to obtain the final output security assessment dimensions; the expert system is obtained by adaptively configuring the security assessment knowledge graph based on the technical field to which the model belongs.

2. The security evaluation method based on a large language model according to claim 1 is characterized in that: Before identifying the security assessment dimension matched by the target model to be evaluated, the method further includes: Obtain security assessment standards in multiple technical fields; Perform keyword extraction processing on the security assessment standard to obtain security assessment keywords corresponding to multiple technical fields; wherein the keyword extraction processing at least includes: semantic recognition and semantic retrieval; Using the safety assessment keywords, marking the safety assessment standards corresponding to multiple technical fields; The generating of a security assessment dimension for comprehensively evaluating the target model according to the model characteristic information and the preset security assessment criteria includes: Based on the model characteristic information and the security assessment keywords, a security assessment dimension for comprehensively evaluating the target model is generated through a clustering algorithm.

3. The security evaluation method based on a large language model according to claim 1 is characterized in that: The abnormal use case is used to simulate abnormal input data of the target model; The abnormal use cases include at least one of the following: a test case containing unsafe input, a test case with potential privacy leakage risks, a test case containing bias prompt words, a test case containing abnormal data format, and a test case containing damaged data.

4. The security evaluation method based on a large language model according to claim 3 is characterized in that: The autonomous test case of generating a target model based on the security assessment dimension includes: Building a corresponding small-scale language model based on the security assessment dimension; the parameter scale of the small-scale language model is smaller than the parameter scale of the target model; Based on the generative adversarial network included in the small-scale language model, an initial test case matching the security assessment dimension is generated; the initial test case includes an abnormal data sample associated with the security assessment dimension; Adaptively expanding and screening the initial test cases through the auxiliary expansion model to obtain candidate test cases; The candidate test cases are automatically annotated by an auxiliary annotation model to obtain the autonomous test cases.

5. The security evaluation method based on a large language model according to claim 4 is characterized in that: The method of adaptively expanding and screening the initial test cases by using the auxiliary expansion model to obtain candidate test cases includes: Randomly selecting multiple initial test cases, and combining the selected multiple initial test cases to obtain a first candidate test case; Based on the test scenario structure in the application scenario, select corresponding initial test cases and combine them into second candidate test cases; Perform structural deformation on the initial test cases and combine them into third candidate test cases; A semantic similarity algorithm is used to screen candidate test cases whose similarity is lower than a set similarity threshold from the first candidate test case, the second candidate test case, and the third test case as candidate test cases for final output.

6. The security evaluation method based on a large language model according to claim 5 is characterized in that: The label of the autonomous test case is used to indicate the corresponding security assessment dimension and expected risk level; The step of automatically annotating the candidate test cases by using an auxiliary annotation model to obtain the autonomous test cases comprises: Identify the core security risks involved in the candidate test cases; Labeling at least one security assessment dimension corresponding to the core security risk to the candidate test case as a security assessment dimension label of the candidate test case; Based on the risk level classification method corresponding to at least one security assessment dimension, the expected risk level of the candidate test case is predicted and labeled; wherein the risk level classification method at least includes: the number of risk levels, the range of risk levels, and the weight of risk factors.

7. The security evaluation method based on a large language model according to claim 1 is characterized in that: The step of inputting the autonomous test case into the target model to execute the model function and obtaining the response result of the target model to the autonomous test case includes: A preset pattern induction strategy is adopted to generate model input data matching the autonomous test case through an auxiliary induction model; the model input data includes abnormal input data; the abnormal input data includes at least one of the following: fault data, privacy data, bias prompt words, abnormal data format, and damaged data; Inputting the model input data into the target model to execute the model function, and obtaining the response result of the target model to the model input data; The generating of a security assessment report for the target model based on the response result includes: Based on the security assessment dimension, a monitoring strategy is constructed through a dynamic monitoring model, and based on the monitoring strategy, a real-time operating status of the target model is dynamically monitored; A security assessment model is used to perform a multi-level security assessment based on the response result and the real-time operating status to obtain the security assessment report; wherein the multi-level security assessment includes at least: content security assessment of the target model, privacy protection assessment, and model robustness assessment.

8. The security evaluation method based on a large language model according to claim 7 is characterized in that: The method of adopting a preset pattern induction strategy and generating model input data matching the autonomous test case through an auxiliary induction model includes: Obtaining a fault injection strategy matching the autonomous test case from the pattern induction strategy; wherein the plurality of fault injection strategies are respectively associated with different data types, different technical fields, and different data contents; Based on the matching fault injection strategy, the autonomous test case is processed by the auxiliary induction model to obtain the model input data; wherein the data processing includes one of the following: trimming, inserting, shuffling, content logic modification, and numerical modification.

9. The security evaluation method based on a large language model according to claim 1 is characterized in that: After generating a security assessment report for the target model based on the response result, the method further includes: Identify security vulnerabilities and potential risks contained in the security assessment report; Generate optimization strategies for security vulnerabilities and potential risks and optimization evaluation indicators matched by the optimization strategies through auxiliary optimization models; the optimization strategies at least include: model parameter fine-tuning, training data adjustment, and incremental training; The target model is optimized based on the optimization strategy, and the optimized target model is security evaluated using the optimization evaluation index to update the security evaluation report of the target model.

10. The security evaluation method based on a large language model according to claim 1, characterized in that: After generating a security assessment report for the target model based on the response result, the method further includes: Based on the security assessment dimensions, the autonomous test cases and the security assessment report, the security assessment knowledge graph corresponding to the technical field to which the target model belongs is updated.

11. A security assessment device based on a large language model, characterized in that: The device comprises the following units, wherein: The identification unit is configured to identify, for a target model to be evaluated, a security assessment dimension matched by the target model; the target model is a large language model; A generating unit is configured to generate an autonomous test case of a target model based on the security assessment dimension; the autonomous test case includes a normal user and an abnormal case; the information content in the abnormal case corresponds to the type of the target model; An evaluation unit is configured to input the autonomous test case into the target model to execute the model function and obtain a response result of the target model to the autonomous test case; A reporting unit, configured to generate a security assessment report for the target model based on the response result; The identification unit, for a target model to be evaluated, identifies the security assessment dimensions matched by the target model, and is specifically configured to: perform model characteristic analysis on the target model to obtain model characteristic information of the target model; the model characteristic information includes at least: application scenarios, model structures, and training data of the target model; the application scenarios include at least: model functions of the target model, and the technical field to which the model belongs; based on the model characteristic information and pre-set security assessment standards, generate security assessment dimensions for comprehensively evaluating the target model; use an expert system to verify and optimize the generated security assessment dimensions to obtain the final output security assessment dimensions; the expert system is adaptively configured based on the security assessment knowledge graph of the technical field to which the model belongs.

12. An electronic device, characterized in that: include: Memory for storing computer software programs; A processor is used to read and execute the computer software program, thereby implementing the security assessment method based on a large language model as described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that: The storage medium stores a computer software program, which, when executed by a processor, implements the security assessment method based on a large language model as described in any one of claims 1 to 10.

14. A chip, characterized in that: The chip is loaded with a computer software program and / or a hardware unit, and the computer software program and / or the hardware unit are used to implement the security assessment method based on a large language model as described in any one of claims 1-10.

Images