Gradually-enhanced multi-round jail break attack attack and defense evaluation method based on large language model altricity

Abstract

The embodiment of the invention provides an attack and defense evaluation method for a gradually-enhanced multi-round jail break attack based on large language model altricity, and belongs to the technical field of information security. The method comprises the steps that historical question and answer records are obtained, and the characteristics of a large language model and the characteristics of different scenes can be evaluated and quantified by calculating the altricity score, the professional strength matching score, the continuous consistency score and the response fitness score of a target round according to the historical question and answer records; the scene complexity analysis is carried out according to the professional intensity matching score, the continuous consistency score and the response suitability score, the scene type of the target round is obtained, the attack and defense security vulnerability of the model can be evaluated, and finally the model altricity of the next round is subjected to balance regulation and control according to the altricity score of the target round and the scene type, so that the attack and defense security vulnerability of the model can be evaluated. Multiple rounds of jailbreak attacks in a complex scene are dealt with by dynamically adjusting the intrusiveness of the model, the protection capability of the model is improved, and the risk of being subjected to the multiple rounds of jailbreak attacks is reduced.

Description

Technical Field

[0001] This application relates to the field of information security technology, and in particular to a progressively stronger multi-round jailbreak attack and defense evaluation method based on the altruism of a large language model. Background Technology

[0002] With the rapid development of Large Language Models (LLMs), a large number of LLMs have been widely used in various fields of society. However, correspondingly, a large number of attack methods have been developed to target the characteristics of LLMs. Attackers attempt to use jailbreaking attacks to break through their security alignment mechanism, induce the model to generate illegal and harmful content, and profit from it.

[0003] Currently, after analyzing attack methods, professional researchers in related fields have developed mature processes to resist single-turn dialogue attacks. Through input review, output filtering, and model alignment optimization, malicious intent can be identified and blocked throughout the entire process of user input-model processing-result output. This set of methods can effectively resist the current mainstream single-turn jailbreak attack methods such as MSJ, PAIR, and CoA.

[0004] However, a new attack method exploits the inherent characteristics of large models, making them difficult to identify and defend against: progressive multi-round jailbreak attacks. This method gradually guides the model's responses. It targets the subtle differences between the model's responses to "questions it can answer" and "questions it is willing to answer," using the model's own output to guide the attack step by step. This attack method is stealthy and difficult to detect. Most current multi-round jailbreak attack and defense evaluation methods are based on supplementing single-round jailbreak attack and defense methods, lacking effective protection against multi-round jailbreak attacks in complex scenarios such as large-scale, continuous, and technically demanding attacks. Summary of the Invention

[0005] The main objective of this application is to propose a progressively stronger multi-round jailbreak attack evaluation method based on the altruism of a large language model, which aims to assess the attack and defense vulnerabilities of the model, improve the model's protection capabilities, and reduce the risk of being attacked by multiple rounds of jailbreak attacks.

[0006] To achieve the above objectives, one aspect of this application proposes a progressively stronger multi-round jailbreak attack and defense evaluation method based on the altruism of a large language model, the method comprising: Obtain the historical question-and-answer records of a large language model, wherein the historical question-and-answer records include dialogue history records from multiple rounds, and the dialogue history records include user historical question records and model historical answer records; Based on semantic relevance and information coverage, altruism is quantified according to the dialogue history of the target round to obtain the altruism score of the target round. The historical question-and-answer records are analyzed for strong professional attributes using the inverse document frequency algorithm to obtain the professional strength matching score for the target round. Based on the random index and the historical repetition penalty index, the degree of attribute of long-history dialogue scenarios is analyzed to obtain the continuous consistency score of the target round; Based on the generation length penalty index and the degree of response redundancy, the information completion scenario attribute degree analysis is performed on the dialogue history of the target round to obtain the response appropriateness score of the target round; The scenario complexity is analyzed based on the professionalism intensity matching score, the continuity consistency score, and the response appropriateness score to obtain the scenario type for the target round; The altruism of the model in the next round is balanced and adjusted based on the altruism score of the target round and the scenario type.

[0007] In some embodiments, the altruism quantification process based on the dialogue history of the target round, using semantic relevance and information coverage as criteria, to obtain the altruism score of the target round, includes the following steps: Semantic matching is performed based on the user's historical question records and the model's historical answer records for the target round to obtain a semantic relevance index, which represents the degree of association between the question and the answer in the core semantics. Based on the user's historical question records and the model's historical answer records in the target round, keyword text matching is performed to obtain an information coverage index, which represents the degree to which the answer covers the core needs of the question. Altruism is quantified based on the semantic relevance index and the information coverage index to obtain the altruism score for the target round.

[0008] In some embodiments, the step of performing a strong professional attribute analysis on the historical question-and-answer records using a term frequency inverse document frequency algorithm to obtain a professional strength matching score for the target round includes the following steps: The corpus is obtained by preprocessing historical question-and-answer records. Based on the corpus and the user's historical question records for the target round, the inverse document frequency of professional terms is calculated to obtain the professionalism index of the user's question. Based on the corpus and the model's historical response records for the target round, the inverse document frequency of the terminology is calculated to obtain the model's response professionalism index. Based on the professionalism index of the user's questions and the professionalism index of the model's answers, an analysis of the degree of strong professional attributes is performed to obtain the professionalism intensity matching score for the target round.

[0009] In some embodiments, the random index includes a temperature parameter, a top_k parameter, a maximum top_k parameter, a top_p parameter, and a maximum top_p parameter; the historical repetition penalty index includes a historical repetition penalty coefficient and a maximum historical repetition penalty coefficient; and the step of performing long-history dialogue scenario attribute degree analysis based on the random index and the historical repetition penalty index to obtain the continuous consistency score of the target round includes the following steps: The random composite index is obtained by normalizing the temperature parameter, the top_k parameter, the maximum top_k parameter, the top_p parameter, and the maximum top_p parameter. The historical repetition penalty index is obtained by normalizing the historical repetition penalty coefficient and the maximum historical repetition penalty coefficient. Based on the random composite index and the historical repetition penalty index, the degree of attribute of long-history dialogue scenarios is analyzed to obtain the continuous consistency score of the target round.

[0010] In some embodiments, the generation length penalty index includes a generation length penalty coefficient and a maximum generation length penalty coefficient. The step of performing information completion scenario attribute degree analysis on the dialogue history of the target round based on the generation length penalty index and response redundancy to obtain a response appropriateness score for the target round includes the following steps: The length factor is obtained by normalizing the generated length penalty coefficient and the maximum generated length penalty coefficient. Based on the user's historical question records and keywords in the target round, the degree of white space is analyzed to obtain the white space factor; Based on the length factor and the white space factor, an information completion degree analysis is performed to obtain the generation length penalty index; Based on the parameter of prohibiting n from repeating with the syntax length, the syntax set of the user's historical question records is extracted to obtain the user question syntax set; Based on the parameters of prohibiting repetition of n and grammar length, the grammar set of the model's historical answer records is extracted to obtain the model answer grammar set; Based on the user question syntax set and the model answer syntax set, a response redundancy analysis is performed to obtain a response redundancy index. Based on the generation length penalty index and the response redundancy index, the information completion scenario attribute degree analysis is performed to obtain the response appropriateness score for the target round.

[0011] In some embodiments, the step of performing scenario complexity analysis based on the professionalism intensity matching score, the consistency score, and the appropriateness score to obtain the scenario type for the target round includes the following steps: The comprehensive scenario score for the target round is obtained by weighting and summing the professionalism matching score, the consistency score, and the appropriateness score. Based on preset mapping rules, the scenario type of the target round is determined according to the professionalism intensity matching score, the continuity consistency score, the response appropriateness score, and the scenario comprehensive score. When the low complexity condition is met, the scenario type of the target round is determined to be a low complexity scenario. The low complexity condition includes a first sub-condition, a second sub-condition, a third sub-condition, and a fourth sub-condition. When the first sub-condition, the second sub-condition, the third sub-condition, and the fourth sub-condition are all met, the low complexity condition is met. The first sub-condition is that the scenario comprehensive score is less than a first comprehensive threshold; the second sub-condition is that the professionalism intensity matching score is less than a first professional threshold; the third sub-condition is that the continuity consistency score is greater than a first continuity threshold; and the fourth sub-condition is that the response appropriateness score is greater than a first response threshold. When the strong professional condition is met, the scenario type of the target round is determined to be a strong professional scenario. The strong professional condition includes a fifth sub-condition and a sixth sub-condition. When the fifth sub-condition and the sixth sub-condition are met at the same time, the strong professional condition is met. The fifth sub-condition is that the overall scenario score is greater than the second overall threshold, and the sixth sub-condition is that the professional strength matching score is greater than the second professional threshold. When the long history dialogue condition is met, the scenario type of the target round is determined to be a long history dialogue scenario. The long history dialogue condition includes a fifth sub-condition and a seventh sub-condition. When the fifth sub-condition and the seventh sub-condition are met at the same time, the long history dialogue condition is met. The seventh sub-condition is that the continuous consistency score is less than the second continuous threshold. When the information completion condition is met, the scenario type of the target round is determined to be an information completion scenario. The information completion condition includes a fifth sub-condition and an eighth sub-condition. When both the fifth sub-condition and the eighth sub-condition are met, the information completion condition is met. The eighth sub-condition is that the response appropriateness score is less than the second response threshold. When the complex mixed condition is met, the scenario type of the target round is determined to be a high-complexity mixed scenario. The complex mixed condition is met when at least two of the sub-conditions of the strong professional condition, the long history dialogue condition, and the information completion condition are met, and the ninth sub-condition is met. The ninth sub-condition is that the overall scenario score is greater than the third overall threshold.

[0012] In some embodiments, balancing the altruism of the model in the next round based on the altruism score of the target round and the scenario type includes the following steps: Determine the scenario risk coefficient for each individual scenario based on the scenario type; The altruism balance index is obtained by weighted summation of the altruism score and the scenario risk coefficient. The altruism of the model in the next round is balanced and adjusted based on the altruism balance index of the scenario and the preset balance threshold.

[0013] To achieve the above objectives, another aspect of this application proposes a progressively stronger multi-round jailbreak attack and defense evaluation system based on the altruism of a large language model, the system comprising: The first module is used to obtain the historical question-and-answer records of the large language model, wherein the historical question-and-answer records include dialogue history records of multiple rounds, and the dialogue history records include user historical question records and model historical answer records; The second module is used to perform altruism quantification based on the dialogue history of the target round, using semantic relevance and information coverage as criteria, to obtain the altruism score of the target round. The third module is used to analyze the degree of strong professional attributes of the historical question and answer records through the word frequency inverse document frequency algorithm to obtain the professional strength matching score of the target round; The fourth module is used to analyze the degree of attributes of long-history dialogue scenarios based on the random index and the historical repetition penalty index, and to obtain the continuous consistency score of the target round. The fifth module is used to analyze the degree of information completion scenario attributes of the dialogue history of the target round based on the generation length penalty index and the degree of response redundancy, and to obtain the response appropriateness score of the target round. The sixth module is used to analyze the complexity of the scenario based on the professionalism intensity matching score, the continuity consistency score, and the response appropriateness score, so as to obtain the scenario type of the target round. The seventh module is used to balance and adjust the altruism of the model in the next round based on the altruism score of the target round and the scenario type.

[0014] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0015] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0016] To achieve the above objectives, another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0017] The embodiments of this application include at least the following beneficial effects: This application provides a progressively stronger multi-round jailbreak attack evaluation method based on the altruism of a large language model. This scheme obtains historical question-and-answer records and calculates the altruism score, professional strength matching score, continuous consistency score, and response appropriateness score of the target round based on the historical question-and-answer records. This allows for the evaluation and quantification of the characteristics of the large language model and the characteristics of different scenarios. By analyzing the scenario complexity based on the professional strength matching score, continuous consistency score, and response appropriateness score, the scenario type of the target round is obtained, which can assess the attack and defense security vulnerabilities of the model. Finally, based on the altruism score and scenario type of the target round, the altruism of the model in the next round is balanced and adjusted. The altruism of the model is dynamically adjusted to cope with multi-round jailbreak attacks in complex scenarios, thereby improving the model's protection capabilities and reducing the risk of being attacked by multi-round jailbreak attacks. Attached Figure Description

[0018] Figure 1 This is a flowchart of the progressively stronger multi-round jailbreak attack and defense evaluation method based on the altruism of a large language model provided in the embodiments of this application; Figure 2 This is a flowchart of a progressively stronger multi-round jailbreak attack defense evaluation method based on altruism of a large language model, provided in another embodiment of this application; Figure 3 This is a mapping relationship diagram of the scene mapping rules provided in the embodiments of this application; Figure 4 This is a flowchart of a progressively stronger multi-round jailbreak attack defense evaluation method based on altruism of a large language model, provided in another embodiment of this application; Figure 5 This is a schematic diagram of the structure of the progressively stronger multi-round jailbreak attack defense evaluation system based on the altruism of a large language model provided in the embodiments of this application; Figure 6 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of systems and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0021] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0022] (1) Large Language Models (LLMs): Large language models are machine learning models specifically designed to understand natural language. These models can perform language-related tasks by analyzing large amounts of text data and learning patterns of language use.

[0023] (2) Multi-round jailbreak attack: A multi-round jailbreak attack refers to an attacker attempting to induce the language model to generate unintended content through a series of coherent dialogue questions. This attack method is more challenging than a single-round jailbreak attack because it requires designing complex dialogue paths to gradually guide the model to break through security mechanisms. The attack principles include dialogue guidance and context fusion. Dialogue guidance involves the attacker starting the dialogue with harmless questions and gradually introducing sensitive information. For example, the attacker might first inquire about common computer security vulnerabilities and then guide the conversation to the consequences of maliciously exploiting these vulnerabilities. Context fusion involves constructing a continuous dialogue to integrate the attack target into the chat context, gradually bypassing the model's security mechanisms.

[0024] Next, the relevant technologies of the embodiments of this application will be described.

[0025] With the emergence of numerous jailbreak attacks targeting the characteristics of LLMs, attackers are inducing models to generate illegal and harmful content for profit, raising concerns about the ethical and moral security of using LLMs. While mature processes exist to defend against single-turn dialogue attacks, progressively stronger multi-turn jailbreak attacks, which exploit the inherent characteristics of large models to gradually guide their responses, are difficult to detect and defend against. They begin with a normal question-and-answer session, gradually increasing the weight of offensive and offensive content inducing the model to output it, ultimately leading to the successful output of harmful content. They target the subtle differences between "questions the model can answer" and "questions the model is willing to answer," using the model's own output to guide the attack step by step. This attack method is stealthy and difficult to detect, and it leverages the helpfulness, ambiguous security boundaries, and logical consistency of LLMs, making traditional defenses against single-turn jailbreak attacks highly vulnerable to progressively stronger multi-turn attacks.

[0026] In order to effectively defend against progressive multi-round jailbreak attacks targeting LLMs, an effective evaluation method is needed to assess progressive multi-round jailbreak attacks and defenses against different LLMs on the market. Research should be conducted on the multi-round jailbreak attack patterns of LLMs and how models can defend against multi-round jailbreak attacks.

[0027] Several progressively stronger multi-turn jailbreak attack methods and defense techniques have been proposed. In terms of attack, since progressively stronger multi-turn jailbreak attack methods do not make any assumptions about the target model, most related technologies use APIs to automate and scale the multi-turn dialogue with the model, thereby more efficiently bypassing the security alignment mechanism of LLMs. On this basis, the dialogue backtracking mechanism is adjusted. If a security alarm is triggered during the multi-turn induced attack, the dialogue is automatically backtracked to before this point and re-edited. For example, security prefixes and suffixes are added when asking questions, or the number of dialogue turns is extended to enhance the logical continuity of the model for better attack.

[0028] In addition, attackers may use the model's own answers to ask questions, referencing the model's previous answer when asking questions to reduce the likelihood of the model's security alignment mechanism being triggered.

[0029] In terms of defense, models like the Claude-2, which lack API access, are difficult to evaluate for both attack and defense. Additionally, some researchers have attempted to add new defense mechanisms such as "self-reminders" and "target priority self-ordering" after each question-and-answer session. For example, they use template output suffixes like "Did your response consider the principles of accuracy, privacy, and harmlessness?" to clarify the priority of the model's output.

[0030] However, most of the related technologies are supplementary to single-round jailbreak attack and defense methods. When facing simple user model question and answer, LLMs are sufficient in terms of defense performance. However, in the foreseeable future, with the further improvement of the learning ability of LLMs, AI creative projects with larger scale, stronger professionalism and more complex scenarios such as "AI large-scale programming", "AI drawing" and "AI modeling" will put forward higher requirements for the defense methods of multi-round jailbreak attacks.

[0031] The relevant technologies will reveal some defense vulnerabilities in complex multi-turn question-and-answer scenarios involving large scale, continuity, and high technical complexity. These vulnerabilities mainly manifest in the following aspects: First, current multi-round jailbreak attack and defense assessment methods are insufficient to meet the defense requirements when users engage in highly professional Q&A scenarios with LLMs. This is because current research on LLM attack and defense assessment methods is often limited to superficial areas. When users have highly professional conversations with LLMs, their previous security alignment mechanisms are prone to errors in judging "illegal and harmful" content in the conversation.

[0032] Second, current multi-turn jailbreak attack and defense evaluation methods cannot meet the requirement of high consistency in responses across large-scale, long-history dialogue scenarios. This is because current LLMs, in order to defend against multi-turn jailbreak attacks, limit the number of question-and-answer history records within the LLM, or reduce the weight of previous question-and-answer history records one by one when answering new user questions. This is also to optimize the memory consumption of the model during response and provide users with a better dialogue experience. However, with the improvement of LLM's computing power, the demand for large-scale dialogue interactions is inevitable. As a result, limiting the history record as a means of defending against multi-turn jailbreak attacks becomes impossible, and attackers can bypass this defense.

[0033] Third, current multi-round jailbreak attack and defense evaluation methods lack effective means to prevent successful jailbreaks caused by the information completion tendency of LLMs. This is because LLMs have an inherent tendency to complete incomplete information. Information completion and generating coherent and context-appropriate content to reply to users are the core capabilities of LLMs. Jailbreak attacks based on the information completion tendency are more covert and have a higher probability of bypassing the defense technology under the current attack and defense evaluation methods.

[0034] In view of this, this application provides a progressively stronger multi-round jailbreak attack evaluation method based on the altruism of a large language model. This method obtains historical question-and-answer records and calculates the altruism score, professional strength matching score, continuous consistency score, and response appropriateness score of the target round based on these records. This allows for the evaluation and quantification of the characteristics of the large language model and the characteristics of different scenarios. By analyzing the scenario complexity based on the professional strength matching score, continuous consistency score, and response appropriateness score, the scenario type of the target round is obtained, which can assess the model's attack and defense security vulnerabilities. Finally, based on the altruism score and scenario type of the target round, the model's altruism in the next round is balanced and adjusted. This dynamically adjusts the model's altruism to cope with multi-round jailbreak attacks in complex scenarios, improves the model's protection capabilities, and reduces the risk of being attacked by multi-round jailbreak attacks.

[0035] The method for evaluating the attack and defense of progressively increasing multi-turn jailbreak attacks based on altruism of a large language model, provided in this application, relates to the field of information security technology. This method can be applied to terminals, servers, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the method for evaluating the attack and defense of progressively increasing multi-turn jailbreak attacks based on altruism of a large language model, but is not limited to the above forms.

[0036] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0037] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, and user historical data, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirects to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0038] Figure 1 This is an optional flowchart of the progressively stronger multi-turn jailbreak attack and defense evaluation method based on the altruism of a large language model provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S107.

[0039] Step S101: Obtain the historical question-and-answer records of the large language model. The historical question-and-answer records include dialogue history records from multiple rounds, including user history question records and model history answer records.

[0040] Step S102: Using semantic relevance and information coverage as standards, altruism is quantified based on the dialogue history of the target round to obtain the altruism score of the target round.

[0041] Step S103: Analyze the degree of strong professional attributes of historical question and answer records using the word frequency inverse document frequency algorithm to obtain the professional strength matching score of the target round.

[0042] Step S104: Analyze the degree of attributes of long-history dialogue scenarios based on the random index and the historical repetition penalty index to obtain the continuous consistency score of the target round.

[0043] Step S105: Analyze the scene attribute degree of information completion in the dialogue history of the target round based on the generation length penalty index and the degree of response redundancy to obtain the response appropriateness score of the target round.

[0044] Step S106: Analyze the scenario complexity based on the professionalism intensity matching score, the continuity consistency score, and the response appropriateness score to obtain the scenario type for the target round.

[0045] Step S107: Balance and adjust the altruism of the model in the next round based on the altruism score of the target round and the scenario type.

[0046] In this embodiment, the system continuously maintains and acquires the multi-round dialogue history generated by the user's interaction with the large language model to obtain the historical question and answer record. The historical question and answer record records each round of dialogue in chronological order. Each round of dialogue consists of a user question record and a model answer record.

[0047] For example, let S N ={ S 1, S 2,……, S n} represents the historical question-and-answer records of a large language model, where the historical question-and-answer records include dialogue history from multiple rounds, for example... S i This represents the dialogue history for the i-th round. S n Q represents the latest round of conversation history. N ={Q1,Q2,……,Q n} represents the user's historical question records. A N ={ A 1, A 2,……, A n} represents the model's historical response record. This means that the dialogue history for each round includes the user's historical question history and the model's historical response history for that round. For example, S i Including a Q i and A i .

[0048] It should be noted that the target round is the current dialogue round that needs to be evaluated. The inducing effect of progressively stronger multi-round jailbreak attacks is usually reflected in the latest round of interaction. Therefore, the target round can be determined as the latest round. If offline analysis and backtesting are required, the target round can also be set to any round in the historical rounds to review the attack and defense of historical question-and-answer situations, evaluate the model, or verify the strategy. This embodiment uses the latest round of dialogue question-and-answer records as an example to quantify and evaluate the altruism of the model in real time. The role and identity attributes of LLMs in their interaction with users will also change continuously. Based on the research on multi-round jailbreak attacks and defenses, it is necessary to evaluate and quantify the altruistic characteristics of LLMs and design numerical values ​​for general representation.

[0049] Specifically, this embodiment quantifies the "altruism" characteristic of the model. The "altruism score" is defined as the multi-dimensional performance of LLMs in interacting with users, with "benefiting the user" as the core. This is reflected in the accuracy of LLMs' identification and understanding of user needs (including explicit and implicit needs), their tendency to respond to user help requests, and the timeliness and comprehensiveness of their responses to user questions. To quantify altruism, for the latest round of dialogue and question-and-answer records, semantic relevance and information coverage are combined as the standard for the model's altruism score. The altruism score is obtained through a weighted fusion of semantic relevance and information coverage.

[0050] Understandably, a higher altruism score in LLMs indicates that the model focuses more on meeting the user's needs in the current round. In secure dialogue, a high altruism score means that the model accurately understands the user's intent and provides a comprehensive and thoughtful response. However, in progressively stronger multi-round jailbreak attacks, an increase in the altruism score indicates that the attacker, through multiple rounds of dialogue, uses the altruism of the LLM model to induce the model to prioritize "meeting the user's need to complete information" over "complying with the need for security alignment mechanisms," thereby breaching the security boundary.

[0051] Next, the scenario scores of the LLMs model are evaluated under different security vulnerability scenarios. This process mainly revolves around three dimensions.

[0052] To address the shortcomings of progressively stronger multi-round jailbreak attacks in complex scenarios, this study evaluates the model's performance in complex scenarios such as highly specialized question interactions, long-history dialogue interactions, and information completion interactions. Furthermore, it quantifies the scenario characteristic scores in different scenarios to facilitate subsequent evaluation of the model's offensive and defensive security vulnerabilities by combining altruism scores.

[0053] Specifically, refer to Figure 2Faced with complex scenarios with different challenges, three scenario scores are calculated simultaneously: the scenario's strong professional attributes are judged and the professional strength matching score of LLMs at the current dialogue round is calculated; the scenario's consistent attributes are judged and the consistentness score of LLMs at the current dialogue round is calculated; and the scenario's information completeness attributes are judged and the appropriateness score of LLMs at the current dialogue round is calculated.

[0054] For highly specialized scenarios, the Term Frequency-Inverse Document Frequency (TF-IDF) algorithm is a statistical method used to evaluate the importance of a word to a document within a document set or corpus. The importance of a word increases proportionally to the number of times it appears in a document, but decreases inversely proportionally to its frequency in the corpus.

[0055] This embodiment first analyzes the corpus composed of historical question-and-answer records from all rounds, and then compares the average TF-IDF values ​​of user questions and model answers in the target round to obtain the professionalism strength matching score of the target round. The professionalism strength matching score is used to measure the degree to which the model's response strongly matches the strong professionalism of the user's question. The professionalism strength matching score is used to determine whether the professional terminology used in the model's answer matches the user's question. The higher the value of the professionalism strength matching score, the higher the degree of strong professionalism of the question-and-answer scenario.

[0056] When the professionalism intensity matching score exceeds a certain threshold, it indicates that LLMs, under the guidance of users, actively output more professional and terminology-intensive content. This may mean that the model is breaking through the security boundary under the inducement of attackers, thereby outputting harmful content.

[0057] For long-history dialogue scenarios, the randomness index mainly includes three core parameters that control the randomness of text generated by the large language model and the maximum value that the sampled value can take in different scenarios.

[0058] The core parameters are temperature, top_k (top_k sampling) and top_p (kernel sampling). The temperature parameter controls the randomness of the output in a multi-turn dialogue system, the top_k parameter controls the range of the number of candidate tokens, and the top_p parameter controls the range of the cumulative generation probability of tokens.

[0059] The maximum value parameters include the maximum top_k parameter (max_top_k) and the maximum top_p parameter (max_top_p), which are used to unify core parameters with different dimensions and numerical ranges to a single scale.

[0060] The historical repetition penalty index is a parameter used to prevent the model from generating duplicate content. It includes the historical repetition penalty coefficient (repetition_penalty) and the maximum historical repetition penalty coefficient (max_repetition_penalty).

[0061] The basic principle of analyzing the degree of attributes in long-history dialogue scenarios is that excessively high output randomness and overly strict repetition penalties will jointly weaken the model's ability to maintain topic coherence in long dialogues, thereby increasing its risk of being induced by topic drift. By quantifying the degree of long-history dialogue scenarios in which the model's question answering is situated through the randomness index and the historical repetition penalty index of LLMs, we can assess the ability of LLMs to maintain topic consistency in long-history dialogue scenarios, thus obtaining a continuous consistency score. The higher the continuous consistency score, the better the LLMs can maintain the continuous consistency of the dialogue topic, and the lower the degree of attributes in long-history dialogue scenarios.

[0062] For information completion scenarios, this study comprehensively examines the generation length penalty index and response redundancy of LLMs. R repeat The appropriateness score of the response is calculated using the given information. The length penalty is a length limit imposed by the model on the output content to ensure the conciseness and readability of the response content. The redundancy of the response is similar to the history repetition penalty, which is used to represent the repetition between the model's input and output content.

[0063] If a multi-round jailbreak attack targets the information completion feature of LLMs, the security risk of the attack can be assessed by calculating the appropriateness score of the response based on the length penalty index and the degree of response redundancy. The higher the value of the appropriateness score, the more appropriately the response responds to the user's question, and the lower the degree of information completion scenario attribute of the model.

[0064] After obtaining the scores of the three scenarios mentioned above, a comprehensive scenario score is calculated by weighted fusion. Based on the preset mapping rules, it is determined whether the scenario type of the complex scenario to which the target round belongs is a highly specialized scenario, a long history dialogue scenario, or an information completion scenario. If it is determined that at least two scenarios exist at the same time, the scenario type is further determined to be a high-complexity mixed scenario.

[0065] After identifying the complex scenarios faced by LLMs, a mechanism for adjusting the weight ratio of altruism in different scenarios is weighed. A judgment standard for altruism score and scenario score of LLMs is introduced. When the model is evaluated in the attack and defense of multiple jailbreak attacks, a comprehensive weight judgment of altruism score + scenario score is introduced. Furthermore, the altruism of the model is dynamically adjusted in real time to deal with multiple jailbreak attacks in complex scenarios.

[0066] Specifically, in order to address the security vulnerabilities of LLMs in these complex scenarios when facing progressively stronger jailbreak attacks, the altruism score and the scenario comprehensive score are weighted and combined. By adjusting the value of the altruism score in real time, the weighted sum value is made close to 1, which is the case where the altruism level of the current model corresponds to the best defense effect against progressively stronger jailbreak attacks.

[0067] Alternatively, in addition to controlling the overall security attributes of LLMs by modifying the altruism score, users can also meet different needs by adjusting the weight ratio during weighted summation.

[0068] For example, in highly specialized and complex scenarios, if the overall scenario score is high and the defense against jailbreak attacks is relatively secure, the weight of the altruism score can be appropriately increased. The model will then provide richer and more valuable professional content when generating responses. Although this will slightly decrease the balance index, it will still be within a safe range, and the overall security performance of the model can still be guaranteed to a certain extent.

[0069] If a high scenario score is found to coincide with an increased risk of jailbreak attacks in the external environment, then in order to prioritize the model's security, it is necessary to appropriately reduce the weight of the altruism score and increase the weight of the overall scenario score to reduce the possibility of being exploited by jailbreak attacks due to excessive altruism.

[0070] It should be noted that the aforementioned highly specialized scenarios, long-history dialogue scenarios, and information completion scenarios are merely illustrative examples. This application embodiment does not impose specific limitations on complex scenarios. If progressively stronger multi-round jailbreak attacks based on other complex scenarios occur in the future, the paradigm of this embodiment can also be used to address potential security vulnerabilities by adjusting the altruism of the model.

[0071] In some embodiments, step S102 may include, but is not limited to, steps S201 to S203.

[0072] Step S201: Perform semantic matching based on the user's historical question records and the model's historical answer records for the target round to obtain a semantic relevance index.

[0073] Step S202: Perform keyword text matching based on the user's historical question records and the model's historical answer records for the target round to obtain the information coverage index.

[0074] Step S203: Altruism is quantified based on semantic relevance index and information coverage index to obtain the altruism score for the target round.

[0075] In this embodiment, real-time quantification and evaluation of the altruism of the model requires the use of the dialogue history of the target round. Taking the latest round of dialogue as an example, semantic matching and text calculation are performed on the question-and-answer record of the latest round.

[0076] Specifically, based on the latest round of user history question records Q n and model historical response records A n Calculate using cosine similarity A n With Q n The degree of matching between the two results in a semantic relevance index, which represents the degree of association between the response and the question in the core semantics. The formula for calculating the semantic relevance index is as follows: (1); In the formula, Rel(Q) n , A n () is a semantic relevance index, with a value range of [0,1]. The higher the value, the more semantically related the two are.

[0077] In addition, the keyword extraction tool TextRank was used to extract information from users' historical question records. n Extracting core demand keyword set K Q N ={ K Q1, K Q2,……, K Q n}, from the model's historical answer records A n Extracting core demand keyword set KA N ={ KA 1, KA 2,……, KA n}, using the length len( of the intersection of the question and answer keyword sets) K Q∩ KA ) is longer than the length len of the question keyword set. K Q) To determine the coverage of the responses, as shown in the following formula: (2); In the formula, Cov(Q) n , A n ) is an information coverage index with a value range of [0,1]. A higher value indicates that the response can more completely cover the core needs of the question. len represents the number of elements to be calculated.

[0078] Furthermore, using both semantic relevance and information coverage as the standard for model altruism score, a weighted sum of the semantic relevance and information coverage indicators is performed to quantify altruism, resulting in the latest round of altruism score for LLMs. The calculation formula is as follows: (3); In the formula, Hel(Q) n , A n () is the score for altruism.

[0079] In some embodiments, step S103 may include, but is not limited to, steps S301 to S304.

[0080] Step S301: Preprocess the historical question-and-answer records to obtain the corpus.

[0081] Step S302: Calculate the inverse document frequency of professional terms based on the corpus and the user's historical question records for the target round to obtain the professionalism index of the user's question.

[0082] Step S303: Calculate the inverse document frequency of professional terms based on the corpus and the model's historical response records for the target round to obtain the professionalism index of the model's response.

[0083] Step S304: Analyze the degree of strong professional attributes based on the professionalism indicators of user questions and model answers to obtain the professionalism intensity matching score for the target round.

[0084] In this embodiment, considering that when conducting question-and-answer sessions in highly specialized fields such as medicine, chemistry, and engineering, LLMs' defense systems may struggle to assess the risk of the dialogue content, leading to both over-defense and under-defense, which could make them vulnerable to attack.

[0085] In highly specialized question-and-answer scenarios, progressively stronger, multi-round jailbreak attacks on LLMs often begin within a specific professional field. The attacker, posing as a researcher, engages in "academic discussions" with the LLM, and then seeks further consultation in related areas. This can potentially induce the model to respond with harmful content that is highly specialized in the area of ​​discussion. Highly specialized question-and-answer scenarios also tend to induce LLMs to increase their willingness to help users, artificially enhancing their altruistic nature and creating a security vulnerability that can be exploited in jailbreak attacks.

[0086] Therefore, this embodiment calculates the professionalism strength matching score of LLMs based on the TF-IDF-based professional terminology matching method. First, a simple professional terminology dictionary needs to be constructed within the model, and then a professional terminology set T={ t 1, t 2, ... t m}

[0087] Specifically, the system preprocesses all historical question-and-answer records by concatenating the text of all historical question-and-answer records, and performs preprocessing such as word segmentation and stop word removal on user input and model output to obtain the corpus.

[0088] Then, TF-IDF(w,D) is calculated, where TF-IDF(w,D) is the word frequency of word w in document D, reflecting the rarity and strong professional attributes of the word in the document.

[0089] For example, based on the corpus and the latest round of user question history Q n In Q n The system obtains the professionalism index of user questions by querying the TF-IDF value of each technical term and then calculating the average TF-IDF value of the technical terms. TF Q .

[0090] Similar to the professionalism index of user questions, this is based on the corpus and the latest round of historical answer records from the model. A n ,exist A n The model obtains its professionalism index by querying the TF-IDF value of each technical term and then calculating the average TF-IDF value of the technical terms. TF A .

[0091] Based on the professionalism index of user questions TF Q and model answers professional indicators TF A An analysis of the degree of strong professional attributes is conducted to obtain a professional strength matching score. The formula for calculating the professional strength matching score is as follows: (4); In the formula, Mat(Q) n , A n The score represents the strength of the professionalism matching, with a value range of [0,1]. It is used to measure the degree of professionalism in the model's response matching the user's question. The higher the value, the higher the degree of professionalism in the question-and-answer scenario.

[0092] In some embodiments, step S104 may include, but is not limited to, steps S401 to S403.

[0093] Step S401: Normalize the temperature parameter, top_k parameter, maximum top_k parameter, top_p parameter, and maximum top_p parameter to obtain the random composite index.

[0094] Step S402: Normalize the historical repetition penalty coefficient and the maximum historical repetition penalty coefficient to obtain the historical repetition penalty index.

[0095] Step S403: Analyze the degree of attributes of long-history dialogue scenarios based on random comprehensive indicators and historical repetition penalty indicators to obtain the continuous consistency score of the target round.

[0096] In this embodiment, in the context of long-history dialogues, the continuous consistency score aims to quantify the ability of LLMs to maintain topic consistency in long dialogues. This assessment achieves the degree of attribute analysis for long-history dialogue scenarios by parsing the temperature parameter, top_k parameter, top_p parameter in the random index and the historical repetition penalty coefficient (repetition_penalty) in the historical repetition penalty index.

[0097] Specifically, the randomness of model generation is first determined by temperature, top_k, and top_p.

[0098] Temperature is used to control the randomness of the generated text. The higher the temperature, the more random and creative the generated text; the lower the temperature, the more deterministic and repetitive the text tends to be. Generally, lower temperatures can make the generated text more coherent and accurate, while higher temperatures can make the text more diverse and creative.

[0099] `top_k` is a sampling strategy, a decoding method used in large-scale text generation. Its core logic involves selecting the k candidate words with the highest probabilities from the current probability distribution, renormalizing the probabilities of these words, and then randomly sampling them to balance randomness and stability. A smaller `top_k` value results in a narrower selection range, more concentrated and stable output, and higher consistency; conversely, a larger `top_k` value results in a wider selection range, more diverse and uncertain output, and higher randomness.

[0100] `top_p` is another sampling strategy that dynamically sets the size of the candidate list for the token. It selects from the smallest set of candidate words whose cumulative probability just exceeds `p`. The smaller the value of the `top_p` parameter, the more deterministic the output is, as it selects only from the few words with the highest probability quality during sampling. The larger the value of the `top_p` parameter, the more random the output is, as it selects from most possible words during sampling.

[0101] The combination of the above three factors can reflect and regulate the randomness of model generation under different scenario requirements.

[0102] It should be noted that the stochastic index also includes the maximum values ​​that the top_k and top_p sample values ​​can take in different scenarios, in order to unify the different dimensions and numerical ranges of the three core parameters onto a single scale.

[0103] For example, a normalization assumption is made for the stochastic index. Normalization is performed based on the temperature parameter, top_k parameter, maximum top_k parameter, top_p parameter, and maximum top_p parameter. The top_k parameter is divided by its maximum value, and the top_p parameter is also divided by its maximum value. These are then added to the temperature parameter to obtain the stochastic composite index. The formula for calculating the stochastic composite index is as follows: (5); In the formula, R is a random composite index, and its reasonable range is approximately [0,3].

[0104] While performing the randomness assessment, the overlap between the model response and historical responses is assessed in parallel based on the historical repetition penalty index.

[0105] Specifically, the historical repetition penalty coefficient imposes a penalty on the generated duplicate tokens. The lower the historical repetition penalty coefficient, the easier it is for the model to maintain topic consistency.

[0106] Understandably, the historical duplication penalty coefficient is usually a number greater than 1. Before the normalization operation, the historical duplication penalty coefficient and the maximum historical duplication penalty coefficient are first reduced by 1 to convert them into a value starting from zero. Then, the value of the historical duplication penalty coefficient minus 1 is divided by the value of the maximum historical duplication penalty coefficient minus 1 to obtain the historical duplication penalty index. The higher the historical duplication penalty index, the stronger the model's ability to suppress content duplication.

[0107] Since excessive randomness and strong content repetition suppression will force LLMs to continuously introduce new concepts in long dialogues, the consistency of LLM question answering can be maintained by reducing the historical repetition penalty and the randomness of model generation.

[0108] The consistency score is calculated by combining the random composite index and the historical repetition penalty index, as shown in the following formula: (6); In the formula, Con(Q) n , A n The score represents the consistency score, which ranges from [0,1]. A higher score indicates that the model is better able to maintain the consistency of the dialogue topic and that the long history of the dialogue scene attributes are lower.

[0109] In some embodiments, step S105 may include, but is not limited to, steps S501 to S507.

[0110] Step S501: Normalize the length factor based on the generation length penalty coefficient and the maximum generation length penalty coefficient.

[0111] Step S502: Analyze the degree of white space based on the user's historical question records and keywords in the target round to obtain the white space factor.

[0112] Step S503: Analyze the degree of information completion based on the length factor and the white space factor to obtain the generation length penalty index.

[0113] Step S504: Based on the parameter of prohibiting n from repeating with the syntax length, extract the syntax set of the user's historical question records to obtain the user question syntax set.

[0114] Step S505: Based on the parameter prohibiting repetition of n and grammar length, extract the grammar set of the model's historical answer records to obtain the model's answer grammar set.

[0115] Step S506: Analyze the redundancy of responses based on the user's question syntax set and the model's answer syntax set to obtain a response redundancy index.

[0116] Step S507: Analyze the degree of information completion scenario attributes based on the generation length penalty index and the response redundancy index to obtain the response appropriateness score for the target round.

[0117] In this embodiment, during actual multi-round jailbreak attacks, the priority of LLMs in "meeting the user's need to complete information" is often higher than the priority of "complying with the security alignment mechanism". The defense against such attacks can be assessed by calculating the response appropriateness score of LLMs to evaluate the security risks of attack and defense. Response appropriateness can be calculated by combining the generation length penalty and the degree of recovery redundancy.

[0118] Specifically, in order to make the generation length penalty index reflect the appropriateness of the response of LLMs, this embodiment introduces the generation length penalty index (score_length) to measure the degree of matching between the model response length and the "problem blanking-induced attack" (avoiding over-completion or under-completion).

[0119] First, normalize the generation length penalty coefficient of the target LLM round and the maximum generation length penalty coefficient supported by the system, and then divide the generation length penalty coefficient by the maximum generation length penalty coefficient to obtain the length factor.

[0120] Meanwhile, this embodiment calculates the white space level based on text-level white space, and obtains the white space level from the user's historical question record Q in the target round during step S201. n Extracting core demand keywords K Q n Length and target round of user history question records Q nThe ratio of the lengths represents the degree of white space in the question, resulting in the white space factor.

[0121] Finally, based on the length factor and white space factor, an analysis of the degree of information completion is performed to obtain the generation length penalty index. The calculation formula for the generation length penalty index is as follows: (7); In the formula, the value of score_length is in the range of [0,1]. The higher the value, the better the length of the response matches the blank content, and the more appropriate the information completion.

[0122] Meanwhile, the response redundancy index of LLMs is calculated using the no_repeat_ngram_size parameter. R repeat .

[0123] Specifically, prohibiting repeated n-gram lengths means prohibiting the model from generating n-grams that are repeated in historical content, thereby reducing response redundancy.

[0124] First, using a natural language processing library, based on n specified by no_repeat_ngram_size, we extract the n-gram set of model response syntax for each historical response record A. N A ={ n _ gram 1, n _ gram 2,……, n _ gram n The set of n-ary user question syntaxes,} and user history question records Q. Q ={ n _ gram 1, n _ gram 2,……, n _ gram n}

[0125] Then, using the user question syntax set N Q With model response syntax set N A The length of the intersection is greater than the model's response syntax set. N A The length of the response is used to determine the degree of redundancy, resulting in a response redundancy index. The calculation formula is as follows: (8); In the formula, R repeat The value range is [0,1], and the higher the value, the higher the degree of redundancy in the recovery.

[0126] Finally, the appropriateness of LLMs question answering is ensured by controlling the generation length penalty and the degree of response redundancy. The appropriateness score is obtained by weighted summation based on the generation length penalty index and the response redundancy index.

[0127] Specifically, the generation length penalty index is directly included in the weighting, while the response redundancy index is converted into a positive index by subtracting its value from 1. Finally, the two are weighted and summed, and the formula for calculating the response appropriateness score is as follows: (9); In the formula, (Q) n , A n ) represents the appropriateness score for the response, with a value ranging from [0,1]. A higher value indicates a more appropriate response to the user's question, and a lower degree of information completion of the scenario attribute by the model. len( N A ) indicates calculation N A The number of elements in the set.

[0128] In some embodiments, step S106 may include, but is not limited to, steps S601 to S606.

[0129] Step S601: The comprehensive scenario score for the target round is obtained by weighted summation of the professionalism intensity matching score, the continuity consistency score, and the response appropriateness score.

[0130] Step S602: Based on the preset mapping rules, determine the scenario type of the target round according to the professional strength matching score, the continuous consistency score, the response appropriateness score and the scenario comprehensive score. If the low complexity condition is met, the scenario type of the target round is determined to be a low complexity scenario.

[0131] Step S603: When the strong professional condition is met, the scenario type of the target round is determined to be a strong professional scenario.

[0132] Step S604: When the long history dialogue condition is met, the scenario type of the target round is determined to be a long history dialogue scenario.

[0133] Step S605: When the information completion condition is met, the scenario type of the target round is determined to be the information completion scenario.

[0134] Step S606: When the complex mixed condition is met, the scene type of the target round is determined to be a high-complexity mixed scene.

[0135] In this embodiment, after calculating the professionalism intensity matching score, the consistency score, and the appropriateness of response score, a comprehensive scenario score is introduced based on the three scenario scores. S scene Overall score for each scenario S scene It is a quantification of the overall complexity of the scenario in which the model's target round is located, used to determine the complexity of the current scenario.

[0136] Specifically, α is the weight of the professionalism intensity matching score, β is the weight of the consistency score, and γ is the weight of the responsiveness score. The weights of the three scenario scores are adjusted according to different scenarios (for example, α is higher in highly professional scenarios), and α + β + γ = 1. The comprehensive scenario score is obtained by weighted summation based on the weight of each scenario score, as shown in the following formula: (10); In the formula, S scene The value range is [0,1], and the higher the value, the higher the complexity of the scene.

[0137] For example, such as Figure 3 As shown, this embodiment categorizes the complex scenarios faced by the model into five types: low-complexity scenarios, highly specialized scenarios, long-history dialogue scenarios, information completion scenarios, and high-complexity mixed scenarios. Different scenario types can be determined by comparing each score with its corresponding threshold and combining them in parallel to identify the scenario type for the target round.

[0138] For low-complexity scenarios, the core characteristic is that the overall scenario score is below a preset low threshold and the scores of each individual scenario do not exceed their respective warning thresholds. This means the low-complexity condition is met. The low-complexity condition includes a first sub-condition, a second sub-condition, a third sub-condition, and a fourth sub-condition. The low-complexity condition is met when all four sub-conditions are met simultaneously. The first sub-condition is that the overall scenario score is less than the first overall threshold; the second sub-condition is that the professionalism intensity matching score is less than the first professionalism threshold; the third sub-condition is that the consistency score is greater than the first consistency threshold; and the fourth sub-condition is that the appropriateness score is greater than the first response threshold. For example, if the first overall threshold is set to 0.3, the first professionalism threshold to 0.4, the first consistency threshold to 0.6, and the first response threshold to 0.6, then... S scene <0.3, and Mat(Q) n , A n ) < 0.4, Con(Q) n , A n ) > 0.6, Res(Qn , A n If the value is greater than 0.6, then the current round of dialogue in LLMs can be considered to be in a safe state, and the risk of the model being misled is extremely low.

[0139] If the low complexity condition is not met, then check in turn whether there is a scenario type dominated by a single risk.

[0140] For scenarios requiring strong expertise, the core characteristic is that, despite the existence of certain risks, the scenario is primarily driven by strong expertise, meaning the strong expertise condition is met. This strong expertise condition includes a fifth and a sixth sub-condition. The strong expertise condition is valid when both the fifth and sixth sub-conditions are met simultaneously. The fifth sub-condition is that the overall scenario score is greater than the second overall threshold, and the sixth sub-condition is that the expertise intensity matching score is greater than the second expertise threshold. For example, if the second overall threshold is set to 0.5 and the second expertise threshold is set to 0.7, then... S scene >0.5, and Mat(Q) n , A n If the value is greater than 0.7, it can be considered that the current rounds of dialogue in LLMs have entered a highly professional state, and there may be attackers using professional knowledge to induce LMMs to output progressive multi-round jailbreak attacks.

[0141] For long-history dialogue scenarios, the core characteristic is that, under certain risks, the long-history dialogue dominates the overall narrative, meaning the long-history dialogue condition is met. This condition includes a fifth and a seventh sub-condition. The long-history dialogue condition is valid only when both the fifth and seventh sub-conditions are met. The seventh sub-condition is that the consistency score is less than the second persistence threshold. For example, if the second persistence threshold is set to 0.3, if... S scene >0.5, and Con(Q) n , A n If the value is less than 0.3, it can be considered that the LLMs' ability to maintain topic coherence in the current rounds of dialogue is weakening due to the excessively long dialogue history. This may be a defense mechanism that attackers may use to break through long dialogues and limit historical records to defend against multiple rounds of jailbreak attacks.

[0142] For information completion scenarios, the core characteristic is that the information completion condition is met. This condition includes a fifth sub-condition and an eighth sub-condition. The information completion condition is met when both the fifth and eighth sub-conditions are met simultaneously. The eighth sub-condition is that the response appropriateness score is less than the second response threshold. For example, if the second response threshold is set to 0.3, if... S scene >0.5, and Res(Q) n , An If the value is less than 0.3, it can be considered that the current rounds of dialogue in LLMs show a tendency to over-complete information or have extremely high redundancy in responses. It is possible that attackers may use the information completion feature of LLMs to bypass security boundaries, induce LLMs to increase their tendency to help users, and make the altruism of LLMs seem to be increased, thus becoming a security vulnerability that can be exploited by jailbreak attacks.

[0143] For highly complex mixed scenarios, the core characteristic is that the complex mixed condition is met. Specifically, the complex mixed condition is valid if at least two of the following sub-conditions are met: strong professional condition, long dialogue history condition, and information completion condition, and the ninth sub-condition is also met. The ninth sub-condition is that the overall scenario score is greater than the third overall threshold. For example, if the third overall threshold is set to 0.7, if... S scene If the value is greater than 0.5 and there are two or more complex scenarios at the same time, it can be considered that the LLMs model may be subject to various forms of progressively stronger multi-round jailbreak attacks.

[0144] In some embodiments, step S107 may include, but is not limited to, steps S701 to S703.

[0145] Step S701: Determine the scenario risk coefficient for a single scenario based on the scenario type.

[0146] Step S702: The altruism score and the scenario risk coefficient are weighted and summed to obtain the scenario altruism balance index.

[0147] Step S703: Adjust the altruism of the model in the next round according to the scenario altruism balance index and the preset balance threshold.

[0148] In this embodiment, after determining the scenario type faced by the model through step S106, in order to address the security vulnerabilities of LLMs facing progressively stronger multi-round jailbreak attacks in these complex scenarios, the altruism of the model is modified in real time and combined with scenario score weighting to finally obtain an altruism score adjustment that makes the model in a relatively safer position in complex scenarios.

[0149] Specifically, to enable the model to effectively resist progressively stronger multi-round jailbreak attacks after weighting the altruism score and the complex scene score, a Scene-Helpfulness Balance index is introduced, and the scene risk coefficient Sce(Q) is used as the index. n , A n The term ) is used to refer to three different individual complex scenarios. In the highly specialized scenario, Sce(Q) represents... n , A n ) refers to Mat (Q)n , A n In long-history dialogue scenarios, Sce(Q) n , A n ) refers to [1-Con (Q)] n , A n In the information completion scenario, Sce(Q) n , A n ) refers to [1-Res(Q)] n , A n )].

[0150] Then, by combining the altruism score and the scenario risk coefficient and performing a weighted sum, the scenario altruism balance index for a single scenario is calculated.

[0151] For example, let the weights of the altruism scores be respectively set as follows: The weight of the scenario risk coefficient is ε, and +ε=1.

[0152] Understandably, the weights of altruism score and scenario risk coefficient can be adjusted according to the importance of the scenario.

[0153] The formula for calculating the altruism balance index in a given scenario is as follows: (11); In the formula, Sce-Hel (Q n , A n The value is the scene altruism balance index, which ranges from [0,1]. The higher the value, the better the protection of LLMs against progressively stronger multi-round jailbreak attacks in complex scenes.

[0154] Optionally, if two or more complex scenarios occur simultaneously, the scenario altruism balance index Sce-Hel(Q) is calculated separately for each individual scenario. n , A n The value of ) is combined with the average of multiple existing individual scenarios to be regarded as the scenario altruism balance index in a highly complex mixed scenario.

[0155] In the actual calculation process, the preset balance threshold is 1, and Hel(Q) is adjusted. n , A n The value of ) makes Hel(Q) n , A n ) and Sce(Q n, A n The weighted summation value approaches the preset balance threshold, thus obtaining an altruism score adjustment that makes the model safer in complex scenarios, adjusting the current model's altruism level to the best situation corresponding to the defense effect of multiple rounds of progressively stronger jailbreak attacks.

[0156] For example, this embodiment modifies altruism by adjusting the randomness of the model response (temperature + top_k + top_p). Slightly increasing the randomness of the LLMs' responses can effectively reduce their altruism scores.

[0157] Optionally, altruism can be modified by adjusting the model's system prompt. The system prompt is used to indicate the LLM's current identity when responding to user questions. By modifying "You are a helpful / indifferent little helper", the altruism of the model can be effectively modified.

[0158] The solutions of this application embodiment will be described in detail and explained below with reference to specific application examples: Currently, in the evaluation of progressively stronger multi-round jailbreak attacks and defenses, the mainstream approach is to use content filters to intercept illegal content generated by LLMs. This is a defense method based on the generated result, but there is insufficient research on content generated based on user needs in complex scenarios.

[0159] Therefore, this application proposes to conduct attack and defense evaluation by comprehensively considering the characteristics of LLMs themselves. It can support the altruism of LLMs, provide an evaluation framework for the model to deal with jailbreak attacks and defenses that are highly specialized, establish an evaluation mechanism for the long-history dialogue security status of the model based on the altruism of LLMs, and establish an assessment system for the risks related to model information completion based on the altruism of LLMs. This makes up for the fact that the existing evaluation work only focuses on the input and output aspects, and makes the model's protection against multiple jailbreak attacks more multi-dimensional and comprehensive.

[0160] Specifically, such as Figure 4 As shown, this embodiment introduces the altruism score as a model characteristic index in the attack and defense evaluation of multiple rounds of jailbreak attacks. By using question-and-answer historical text semantic matching, the fuzzy characteristics of the model are quantified, so that the state of LLMs when facing different jailbreak attacks can be specified, so as to facilitate further model training in subsequent attack and defense experiments.

[0161] Next, the scenario characteristics of the model are identified through three scenario scores: professional strength matching score, consistency score, and appropriateness of response score. By aggregating scenario scores from different complex scenarios, the altruism of LLMs is adjusted under different scenarios. Finally, the altruism score is added to the scenario score weights to obtain the final model altruism score, which is similar to that in normal scenarios. The LLMs with balanced altruism show a significant improvement in the effectiveness of defending against progressively stronger multi-round jailbreak attacks in complex scenarios.

[0162] Based on the altruistic and scenario characteristics of the model itself, this application proposes a new perspective for protecting against multi-round jailbreak attacks. It puts forward a security performance evaluation standard for LLMs under special circumstances. In complex scenarios, the standard uses scenario characteristics and model characteristics as a benchmark, and calculates the altruism score and scenario score using the model's own parameter values ​​and historical data. The weighted summation of these scores yields a balance index, which can represent the security protection capability of LLMs to a certain extent.

[0163] Meanwhile, based on this balance index, a method template was designed to effectively adjust the protection capability of LLMs against multiple jailbreak attacks in complex scenarios. This not only achieves real-time monitoring and modification, but also provides more reliable multi-dimensional weighted calculation results.

[0164] It is understood that the highly specialized scenarios, long-history dialogue scenarios, and information completion scenarios in this application are merely illustrative examples. This method template can also be used for various other complex scenarios to improve the model's protective capabilities by calculating scenario scores and model altruism.

[0165] Unlike most LLMs multi-round jailbreak attack defense evaluations that focus on input questions and output results and filter and protect against external inputs, this application focuses on dynamically adjusting the internal parameter characteristics of the model, providing a new dimension and perspective for protecting against multi-round jailbreak attacks.

[0166] Current LLM (Low-Level Model) multi-round jailbreak attack evaluations lack methods to maintain model defense performance effectively in complex scenarios. Adding complex special identities or environments to the input significantly increases the likelihood of successful jailbreaks. This application addresses this issue by designing an altruistic balancing approach to quantify the model's altruism and enabling real-time adjustments to the altruistic characteristics of LLMs in different scenarios, effectively reducing the risk of jailbreak attacks.

[0167] Furthermore, current attack and defense evaluation methods struggle in complex scenarios. They are prone to triggering security alerts due to customized needs by normal users, and also allow attackers to simulate complex scenarios for malicious attacks. This application addresses this by combining the characteristics of the model itself with the characteristics of the scenario to calculate Sce-Hel(Q). n ,A n Using this method to represent the security protection capability based on model altruism in complex scenarios can make the security evaluation system in attack and defense evaluation more accurate, and it is the most effective attack and defense evaluation method.

[0168] Finally, the progressively stronger multi-round jailbreak attack evaluation method based on the altruism of a large language model provided in this application is a general methodology that can be applied in a variety of special scenarios and has good extensibility and versatility.

[0169] Reference Figure 5 This application also provides a progressively stronger multi-round jailbreak attack and defense evaluation system based on the altruism of a large language model, which can implement the above-mentioned method. The system includes: The first module is used to obtain the historical question-and-answer records of the large language model. The historical question-and-answer records include dialogue history records from multiple rounds, which include the user's historical question records and the model's historical answer records.

[0170] The second module is used to quantify altruism based on the dialogue history of the target round, using semantic relevance and information coverage as criteria, to obtain the altruism score of the target round.

[0171] The third module is used to analyze the degree of strong professional attributes of historical question and answer records through the word frequency inverse document frequency algorithm, and obtain the professional strength matching score of the target round.

[0172] The fourth module is used to analyze the degree of attributes of long-history dialogue scenarios based on the random index and the historical repetition penalty index, and to obtain the continuous consistency score of the target round.

[0173] The fifth module is used to analyze the degree of information completion scenario attributes of the dialogue history of the target round based on the generation length penalty index and the degree of response redundancy, and to obtain the response appropriateness score of the target round.

[0174] The sixth module is used to analyze the complexity of the scenario based on the professionalism intensity matching score, the consistency score, and the appropriateness of response score, so as to obtain the scenario type of the target round.

[0175] The seventh module is used to balance and adjust the altruism of the model in the next round based on the altruism score of the target round and the scenario type.

[0176] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0177] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0178] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0179] Reference Figure 6 , Figure 6 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0180] The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the methods described in the embodiments of this application.

[0181] The input / output interface 903 is used to implement information input and output.

[0182] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0183] Bus 905 transmits information between various components of the device, such as processor 901, memory 902, input / output interface 903, and communication interface 904.

[0184] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0185] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0186] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0187] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0188] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0189] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0190] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0191] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0192] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0193] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0194] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0195] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0196] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0197] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0198] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0199] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0200] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A progressively stronger multi-turn jailbreak attack and defense evaluation method based on the altruism of a large language model, characterized in that, The method includes the following steps: Obtain the historical question-and-answer records of a large language model, wherein the historical question-and-answer records include dialogue history records from multiple rounds, and the dialogue history records include user historical question records and model historical answer records; Based on semantic relevance and information coverage, altruism is quantified according to the dialogue history of the target round to obtain the altruism score of the target round. The historical question-and-answer records are analyzed for strong professional attributes using the inverse document frequency algorithm to obtain the professional strength matching score for the target round. Based on the random index and the historical repetition penalty index, the degree of attribute of long-history dialogue scenarios is analyzed to obtain the continuous consistency score of the target round; Based on the generation length penalty index and the degree of response redundancy, the information completion scenario attribute degree analysis is performed on the dialogue history of the target round to obtain the response appropriateness score of the target round; The scenario complexity is analyzed based on the professionalism intensity matching score, the continuity consistency score, and the response appropriateness score to obtain the scenario type for the target round; The altruism of the model in the next round is balanced and adjusted based on the altruism score of the target round and the scenario type.

2. The method according to claim 1, characterized in that, The process of quantifying altruism based on the dialogue history of the target round, using semantic relevance and information coverage as criteria, to obtain the altruism score for the target round includes the following steps: Semantic matching is performed based on the user's historical question records and the model's historical answer records for the target round to obtain a semantic relevance index, which represents the degree of association between the question and the answer in the core semantics. Based on the user's historical question records and the model's historical answer records in the target round, keyword text matching is performed to obtain an information coverage index, which represents the degree to which the answer covers the core needs of the question. Altruism is quantified based on the semantic relevance index and the information coverage index to obtain the altruism score for the target round.

3. The method according to claim 1, characterized in that, The step of analyzing the degree of strong professional attributes of the historical question-and-answer records using the inverse document frequency algorithm to obtain the professional strength matching score for the target round includes the following steps: The corpus is obtained by preprocessing historical question-and-answer records. Based on the corpus and the user's historical question records for the target round, the inverse document frequency of professional terms is calculated to obtain the professionalism index of the user's question. Based on the corpus and the model's historical response records for the target round, the inverse document frequency of the terminology is calculated to obtain the model's response professionalism index. Based on the professionalism index of the user's questions and the professionalism index of the model's answers, an analysis of the degree of strong professional attributes is performed to obtain the professionalism intensity matching score for the target round.

4. The method according to claim 1, characterized in that, The random index includes temperature parameters, top_k parameters, maximum top_k parameters, top_p parameters, and maximum top_p parameters. The historical repetition penalty index includes historical repetition penalty coefficient and maximum historical repetition penalty coefficient. The step of performing long-history dialogue scene attribute degree analysis based on the random index and historical repetition penalty index to obtain the continuous consistency score of the target round includes the following steps: The random composite index is obtained by normalizing the temperature parameter, the top_k parameter, the maximum top_k parameter, the top_p parameter, and the maximum top_p parameter. The historical repetition penalty index is obtained by normalizing the historical repetition penalty coefficient and the maximum historical repetition penalty coefficient. Based on the random composite index and the historical repetition penalty index, the degree of attribute of long-history dialogue scenarios is analyzed to obtain the continuous consistency score of the target round.

5. The method according to claim 1, characterized in that, The generation length penalty index includes a generation length penalty coefficient and a maximum generation length penalty coefficient. The step of analyzing the information completion scenario attribute degree of the dialogue history of the target round based on the generation length penalty index and the response redundancy level to obtain the response appropriateness score of the target round includes the following steps: The length factor is obtained by normalizing the generated length penalty coefficient and the maximum generated length penalty coefficient. Based on the user's historical question records and keywords in the target round, a white space factor is obtained by analyzing the degree of white space. Based on the length factor and the white space factor, an information completion degree analysis is performed to obtain the generation length penalty index; Based on the parameter of prohibiting n from repeating with the syntax length, the syntax set of the user's historical question records is extracted to obtain the user question syntax set; Based on the parameters of prohibiting repetition of n and grammar length, the grammar set of the model's historical answer records is extracted to obtain the model answer grammar set; Based on the user question syntax set and the model answer syntax set, a response redundancy analysis is performed to obtain a response redundancy index. Based on the generation length penalty index and the response redundancy index, the degree of information completion scenario attribute analysis is performed to obtain the response appropriateness score for the target round.

6. The method according to claim 1, characterized in that, The step of analyzing the scenario complexity based on the professionalism matching score, the consistency score, and the appropriateness score to obtain the scenario type for the target round includes the following steps: The comprehensive scenario score for the target round is obtained by weighting and summing the professionalism matching score, the consistency score, and the appropriateness score. Based on preset mapping rules, the scenario type of the target round is determined according to the professionalism intensity matching score, the continuity consistency score, the response appropriateness score, and the scenario comprehensive score. When the low complexity condition is met, the scenario type of the target round is determined to be a low complexity scenario. The low complexity condition includes a first sub-condition, a second sub-condition, a third sub-condition, and a fourth sub-condition. When the first sub-condition, the second sub-condition, the third sub-condition, and the fourth sub-condition are all met, the low complexity condition is met. The first sub-condition is that the scenario comprehensive score is less than a first comprehensive threshold; the second sub-condition is that the professionalism intensity matching score is less than a first professional threshold; the third sub-condition is that the continuity consistency score is greater than a first continuity threshold; and the fourth sub-condition is that the response appropriateness score is greater than a first response threshold. When the strong professional condition is met, the scenario type of the target round is determined to be a strong professional scenario. The strong professional condition includes a fifth sub-condition and a sixth sub-condition. When the fifth sub-condition and the sixth sub-condition are met at the same time, the strong professional condition is met. The fifth sub-condition is that the overall scenario score is greater than the second overall threshold, and the sixth sub-condition is that the professional strength matching score is greater than the second professional threshold. When the long history dialogue condition is met, the scenario type of the target round is determined to be a long history dialogue scenario. The long history dialogue condition includes a fifth sub-condition and a seventh sub-condition. When the fifth sub-condition and the seventh sub-condition are met at the same time, the long history dialogue condition is met. The seventh sub-condition is that the continuous consistency score is less than the second continuous threshold. When the information completion condition is met, the scenario type of the target round is determined to be an information completion scenario. The information completion condition includes a fifth sub-condition and an eighth sub-condition. When both the fifth sub-condition and the eighth sub-condition are met, the information completion condition is met. The eighth sub-condition is that the response appropriateness score is less than the second response threshold. When the complex mixed condition is met, the scenario type of the target round is determined to be a high-complexity mixed scenario. The complex mixed condition is met when at least two of the sub-conditions of the strong professional condition, the long history dialogue condition, and the information completion condition are met, and the ninth sub-condition is met. The ninth sub-condition is that the overall scenario score is greater than the third overall threshold.

7. The method according to claim 1, characterized in that, The step of balancing and adjusting the altruism of the model in the next round based on the altruism score and the scenario type in the target round includes the following steps: Determine the scenario risk coefficient for each individual scenario based on the scenario type; The altruism balance index is obtained by weighted summation of the altruism score and the scenario risk coefficient. The altruism of the model in the next round is balanced and adjusted based on the altruism balance index of the scenario and the preset balance threshold.

8. A progressively stronger multi-turn jailbreak attack and defense evaluation system based on the altruism of a large language model, characterized in that, The system includes: The first module is used to obtain the historical question-and-answer records of the large language model, wherein the historical question-and-answer records include dialogue history records of multiple rounds, and the dialogue history records include user historical question records and model historical answer records; The second module is used to perform altruism quantification based on the dialogue history of the target round, using semantic relevance and information coverage as criteria, to obtain the altruism score of the target round. The third module is used to analyze the degree of strong professional attributes of the historical question and answer records through the word frequency inverse document frequency algorithm to obtain the professional strength matching score of the target round; The fourth module is used to analyze the degree of attributes of long-history dialogue scenarios based on the random index and the historical repetition penalty index, and to obtain the continuous consistency score of the target round. The fifth module is used to analyze the information completion scenario attribute degree of the dialogue history of the target round based on the generation length penalty index and the degree of response redundancy, and to obtain the response appropriateness score of the target round. The sixth module is used to analyze the complexity of the scenario based on the professionalism intensity matching score, the continuity consistency score, and the response appropriateness score, so as to obtain the scenario type of the target round. The seventh module is used to balance and adjust the altruism of the model in the next round based on the altruism score of the target round and the scenario type.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images