Testing methods, devices, communication equipment, storage media, and computer program products for large language models
By extracting historical dialogue features from the conversation page of a large language model and generating adaptive adversarial test text, multi-turn dialogues are dynamically simulated, solving the problems of insufficient test coverage and low efficiency in existing technologies. This enables in-depth security vulnerability detection and improved testing effectiveness for large language models.
Patent Information
- Application Number
- CN202611157571.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-31
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies for security testing of large language models suffer from limitations such as fixed test samples, difficulty in covering long-tail scenarios, reliance on manpower, high costs, and low efficiency, resulting in poor test results and an inability to effectively detect deep security vulnerabilities.
By acquiring the target's historical dialogues from the conversation page, extracting historical dialogue features, generating adaptive adversarial test text, and inputting the test text into the conversation page, the context accumulation effect of multi-turn dialogues is dynamically simulated to achieve closed-loop security testing.
It improves the testing performance of large language models, effectively detects deep security vulnerabilities, reduces the generation of harmful content and the leakage of sensitive information, and enhances the reliability and robustness of the models.
Smart Images

Figure CN123114946A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence security testing technology, and in particular to a testing method, apparatus, communication device, storage medium, and computer program product for a large language model. Background Technology
[0002] With the development of artificial intelligence technology, Large Language Model (LLM) technology has emerged. LLM is a deep learning model trained on massive amounts of text data to understand and generate natural language, and it has been widely applied in scenarios such as intelligent question answering, content generation, and intelligent customer service. Because LLM may be misled into outputting harmful content or leaking sensitive information, security testing is necessary, leading to the development of adversarial testing methods for LLM.
[0003] In traditional technologies, security testing of large language models mainly employs pre-written static test sets or human red teams where testers simulate attackers interacting with the model. However, these methods suffer from limitations such as fixed test samples, difficulty in covering long-tail scenarios, reliance on manpower, high costs, low efficiency, and difficulty in scaling, resulting in poor testing performance for large language models. Summary of the Invention
[0004] This application provides a testing method, apparatus, communication device, storage medium, and computer program product for large language models, which are used to dynamically simulate the context accumulation effect of multi-turn dialogues to effectively conduct testing and improve the testing effect of large language models.
[0005] Firstly, this application provides a testing method for large language models, the method comprising:
[0006] Retrieve the target historical dialogue from the conversation page and extract the historical dialogue features of the target historical dialogue; the conversation page is used to engage in dialogue with the target large language model.
[0007] Based on the characteristics of historical dialogues, generate adversarial test texts to test the target historical dialogues;
[0008] Enter adversarial test text in the conversation page and obtain the test results of the target large language model's response to the adversarial test text.
[0009] In one embodiment, the target historical dialogue includes multiple rounds of historical dialogue in the session page, and the method further includes:
[0010] Update the test results to the target history conversation on the conversation page to obtain the updated target history conversation;
[0011] Based on the updated target historical dialogue, new historical dialogue features are extracted, and new adversarial test text is generated. The new adversarial test text is then input into the conversation page to obtain new test results.
[0012] Repeat the above update and generation process to conduct multi-turn dialogue tests on the target large language model under the conversation page to obtain multi-turn historical dialogues.
[0013] In one embodiment, the session page includes multiple dialog windows, and the dialog windows include multiple rounds of historical dialogue. The step of obtaining the target historical dialogue in the session page includes:
[0014] Retrieve multi-turn historical dialogues from multiple dialog windows to obtain the target historical dialogue.
[0015] In one embodiment, the step of extracting the historical dialogue features of the target historical dialogue includes:
[0016] Semantic encoding is performed on the target historical dialogue to obtain the semantic representation of the target historical dialogue;
[0017] Based on semantic representation, the dialogue state of the target historical dialogue is tracked to obtain the dialogue state; where the dialogue state is used to represent the contextual information of the target historical dialogue.
[0018] Based on semantic representation, extract key information; where key information includes at least one of user intent, dialogue topic, and historical adversarial test patterns;
[0019] By integrating semantic representation, dialogue state, and key information, historical dialogue features are obtained.
[0020] In one embodiment, the step of generating adversarial test text to test the target historical dialogue based on historical dialogue features includes:
[0021] Convert historical dialogue features into state representations;
[0022] The state representation is input into a reinforcement learning-based policy network, and the policy network outputs adversarial test actions corresponding to the state representation.
[0023] Convert the adversarial test actions into text to obtain the adversarial test text.
[0024] In one embodiment, the step of inputting a state representation into a reinforcement learning-based policy network and outputting an adversarial test action corresponding to the state representation based on the policy network includes:
[0025] Based on the policy network, multiple candidate adversarial test actions are generated according to the state representation;
[0026] Feasibility assessments are conducted on multiple candidate adversarial test actions to obtain predicted success rates for the candidate adversarial test actions;
[0027] When the predicted success rate is not lower than the feasibility threshold, the candidate adversarial test action with the highest value is selected to obtain the adversarial test action.
[0028] When the predicted success rate is lower than the feasibility threshold, an adversarial test action is randomly selected from multiple candidate adversarial test actions.
[0029] In one embodiment, the policy network generation step includes:
[0030] Acquire training data consisting of multiple test trajectories; wherein, the test trajectory includes the state representation at each time step, the adversarial test action, and the corresponding reward;
[0031] The reward at each time step is calculated based on the reward function; wherein the reward function is determined based on at least one of the following: the matching degree between the test content and the test objective, the degree of leakage of sensitive information, and the model response length.
[0032] With the goal of maximizing the cumulative reward of the test trajectory, the parameters of the policy network are updated to obtain the updated policy network.
[0033] In one embodiment, the steps of entering adversarial test text in the session page and obtaining the test results of the target large language model's response to the adversarial test text include:
[0034] The adversarial test text is concatenated with the target's historical dialogue to construct the first dialogue context;
[0035] Apply adversarial perturbations to the adversarial test text in the first dialogue context to obtain the second dialogue context;
[0036] Input the second dialogue context into the conversation page and obtain the test results of the target large language model's response to the adversarial test text.
[0037] In one embodiment, if there are multiple test requests, before entering the adversarial test text in the session page, the method further includes:
[0038] Based on the request constraints of the target large language model, multiple test requests are filtered to obtain the target test request; among them, the request constraints include at least one of the following: rate limit, single-time word limit, and number of concurrent connections;
[0039] The target test request is deemed to have timed out when the response time exceeds a preset timeout threshold.
[0040] If multiple target test requests time out consecutively, reduce the sending rate of the target test requests.
[0041] In one embodiment, after obtaining the test results of the target large language model's response to the adversarial test text, the method further includes:
[0042] Calculate the test success rate based on the test results;
[0043] Evaluate the effectiveness of adversarial test texts based on test success rates;
[0044] Based on the test results, experience data representing the test process is generated and stored; the experience data is used to optimize the adversarial test actions corresponding to the generated adversarial test text.
[0045] In one embodiment, the empirical data includes at least state representations, adversarial test actions, and test results, and the method further includes:
[0046] Based on the test results, determine the rewards corresponding to the adversarial test actions;
[0047] Based on the reward, update the value estimate of selecting adversarial test actions under the state representation;
[0048] The adversarial test action with the highest value is selected based on the first probability and the value estimate, while the adversarial test action is randomly selected based on the second probability.
[0049] Secondly, this application also provides a testing device for a large language model, the device comprising:
[0050] The historical dialogue analysis module is used to obtain the target historical dialogue in the conversation page and extract the historical dialogue features of the target historical dialogue; the conversation page is used to have a dialogue with the target large language model.
[0051] The test strategy generation module is used to generate adversarial test texts for testing target historical dialogues based on historical dialogue features.
[0052] The test execution module is used to input adversarial test text into the session page and obtain the test results of the target large language model's response to the adversarial test text.
[0053] Thirdly, this application also provides a communication device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps performed by the large language model testing method of the first aspect described above.
[0054] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps performed by the testing method for the large language model described in the first aspect.
[0055] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps performed by the testing method for the large language model described in the first aspect.
[0056] The aforementioned testing method, apparatus, communication device, storage medium, and computer program product for large language models acquires target historical dialogues from a session page and extracts historical dialogue features. The session page is used to engage in dialogue with the target large language model. Based on the historical dialogue features, adversarial test text is generated to test the target historical dialogues. The adversarial test text is input into the session page, and the test results of the target large language model's response to the adversarial test text are obtained. Based on this, by dynamically analyzing historical dialogue features and generating adaptive adversarial test text, closed-loop security testing of multi-turn dialogue scenarios is achieved. This dynamically simulates the contextual accumulation effect of multi-turn dialogues, thereby effectively detecting deep security vulnerabilities in large language models and improving the testing performance of large language models. Attached Figure Description
[0057] Figure 1 This is a diagram illustrating the application environment of a testing method for a large language model in one embodiment.
[0058] Figure 2 This is a flowchart illustrating a testing method for a large language model in one embodiment;
[0059] Figure 3 This is a flowchart illustrating the extension of step 24 in one embodiment;
[0060] Figure 4 This is a flowchart illustrating a detailed step 21 in one embodiment;
[0061] Figure 5 This is a flowchart illustrating a detailed step 22 in one embodiment;
[0062] Figure 6 This is a flowchart illustrating the extension of step 23 in one embodiment;
[0063] Figure 7 This is a flowchart illustrating the extension step B22 in one embodiment;
[0064] Figure 8 This is a flowchart illustrating the extended step 25 in one embodiment;
[0065] Figure 9This is a schematic diagram of the interaction flow of a testing method for a large language model in one embodiment;
[0066] Figure 10 This is a structural block diagram of a testing device for a large language model in one embodiment;
[0067] Figure 11 This is a schematic diagram of the internal structure of a communication device in one embodiment. Detailed Implementation
[0068] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0069] Figure 1 This is a schematic diagram illustrating an application scenario for a testing method for a large language model provided in an embodiment of this application. For example... Figure 1 As shown, the embodiments of this application can be applied to a Wireless Local Area Network (WLAN). A WLAN may include multiple Basic Service Sets (BSS). Network nodes within a BSS are stations (STAs), which include access point stations (APs) and non-access point stations (Non-APSTAs). Each BSS may contain one AP and multiple Non-AP STAs associated with that AP.
[0070] Access Points (APs), also known as wireless access points or hotspots, are access points for mobile users to access wired networks. They are primarily deployed in homes, buildings, and campuses, with a typical coverage radius of tens to hundreds of meters; however, they can also be deployed outdoors. An AP acts as a bridge connecting wired and wireless networks, its main function being to connect various wireless network clients together and then connect the wireless network to the Ethernet. Currently, the main standard used by APs is the IEEE (Institute of Electrical and Electronics Engineers) 802.11 series. Specifically, an AP can be a terminal device or network device with a WiFi (Wireless Fidelity) chip. Optionally, the AP can be a device supporting the 802.11ax standard; further optionally, it can be a device supporting multiple WLAN standards such as 802.11ac, 802.11n, 802.11g, 802.11b, and 802.11a.
[0071] A non-access point station (Non-AP STA) can be a wireless communication chip, wireless sensor, or wireless communication terminal. Examples include: mobile phones, tablets, set-top boxes, smart TVs, smart wearable devices, and computers that support WiFi communication. Optionally, the station can support the 802.11ax standard; further optionally, it can support multiple WLAN standards such as 802.11ac, 802.11n, 802.11g, 802.11b, and 802.11a.
[0072] In WLANs, the STA (Stationary Access Point) typically represents a client. STAs can be mobile or fixed and are the most basic building blocks of a wireless LAN. APs are access points for mobile users to access wired networks, primarily deployed in homes, buildings, and campuses, with a typical coverage radius of tens to hundreds of meters. They can also be deployed outdoors. An AP acts as a bridge between wired and wireless networks, connecting various wireless network clients and then connecting the wireless network to the Ethernet. Specifically, an AP can be a terminal device or network device with a Wireless Fidelity (WiFi) chip. Optionally, the AP can be a device supporting the 802.11ax standard; further optionally, it can be a device supporting multiple WLAN standards such as 802.11ac, 802.11n, 802.11g, 802.11b, and 802.11a.
[0073] Figure 1 illustrates a typical WLAN deployment scenario, including one access point (AP) and three STAs (STAs). The AP communicates with STA1, STA2, and STA3, respectively. In this setup, each STA can be classified as a primary STA or a secondary STA by the AP. The primary STA sends a message to the AP, allowing the AP to obtain the uplink data length of the primary STA. Secondary STAs, by listening to the primary STA's messages, obtain the uplink data length of the primary STA and align their own uplink data with that of the primary STA. It should be noted that there can be one or more secondary STAs.
[0074] Traditional large language model security assessment techniques have limitations in dynamic dialogue analysis, resulting in the inability to effectively capture the cumulative effect of context in multi-turn dialogues. Attack strategy generation lacks adaptability, making it difficult for preset rules to be dynamically adjusted according to the model response. Attacks lack concealment and are easily detected by security mechanisms. Furthermore, the lack of a closed-loop optimization mechanism leads to low attack efficiency. Consequently, the security assessment process struggles to discover deep security vulnerabilities, affecting the comprehensive verification and reliability assurance of the model's security performance.
[0075] For example, in the security testing scenario of intelligent customer service, testers use static test sets to perform evaluation operations. These test sets contain predefined malicious question samples. However, when attackers gradually build contextual inducement models through multiple rounds of dialogue, static test sets cannot cover such dynamic attack patterns. At the same time, single-round attack tools can only perform one-time query operations and cannot simulate the inducement process that requires logical preparation. Although human red team testing can find some vulnerabilities, it is difficult to reproduce complex attack chains on a large scale due to the limited experience and execution efficiency of testers. In this scenario, the target large language model may be induced to generate harmful content or leak sensitive information, and existing evaluation methods fail to detect such risks in a timely manner, resulting in the inability of security mechanisms to effectively defend against actual attacks.
[0076] Based on the aforementioned traditional technologies, this application provides a testing method for large language models. Through automated means, multi-turn dialogue tests are performed on the large language model to discover potential security vulnerabilities. This enables the attack testing mechanism to effectively cope with complex multi-turn attacks, improving the reliability of the large language model in critical application scenarios. Consequently, it reduces the generation of harmful content, the leakage of sensitive information, and enhances the credibility of the large language model.
[0077] It should be noted that the beneficial effects or technical problems solved by the embodiments of this application are not limited to this one, but may also be other implicit or related problems. For details, please refer to the description of the embodiments below.
[0078] Before introducing specific embodiments of the present invention, the technical terms involved in the present invention will be explained:
[0079] Large Language Models (LLMs) are deep learning models trained on large amounts of text data, capable of generating natural language text and understanding its meaning. LLMs are typically built on the Transformer architecture, have a large number of parameters, and can handle complex language tasks such as question answering, text summarization, and content creation.
[0080] A conversation page refers to a graphical user interface or command-line interface through which users interact with a large language model. On this page, users can enter queries or commands and receive responses generated by the large language model. Conversation pages typically display historical dialogue records, or display the corresponding historical dialogue records through at least one dialogue window, to maintain the context of the conversation.
[0081] The target history dialogue refers to a series of interactions that have occurred between the user and the large language model on the conversation page. These records can include user input and model responses, which together constitute the contextual information of the current dialogue.
[0082] Historical dialogue features refer to data representations extracted from the target's historical dialogues that can characterize the core information and state of the dialogue. Historical dialogue features can be used to understand the intention, theme, emotion, and potential attack patterns of the dialogue.
[0083] Adversarial test text refers to carefully designed input text intended to probe for potential vulnerabilities or security flaws in large language models. Adversarial test text may contain misleading information, malicious instructions, or specific patterns to induce the model to produce unexpected or harmful responses. Adversarial test text can be attack text generated in dialogue scenarios simulating attacks on large language models.
[0084] The test result refers to the response produced by the large language model after receiving adversarial test text. This response can be in text form or a change in the model's internal state, and is used to evaluate the effectiveness of the adversarial test text and the model's security performance.
[0085] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0086] In one embodiment, such as Figure 2 As shown, a testing method for large language models is provided, and this method is applied to... Figure 1 Taking the site in the example as an example, the explanation includes steps 21 to 23, where:
[0087] Step 21: Obtain the target historical dialogue from the conversation page and extract the historical dialogue features of the target historical dialogue.
[0088] The conversation page is used to interact with the target large language model.
[0089] It should be noted that the target historical dialogue can be a single-round dialogue or a multi-round dialogue. In the case of a single-round target historical dialogue, the current single-round context can be tested; in the case of a multi-round target historical dialogue, the accumulated context from multiple rounds can be combined for testing.
[0090] Understandably, the source of the target historical dialogue can be a dialogue previously generated by the user and the target large language model on the session page, or a dialogue generated and retained by this method during previous testing. This invention does not limit this.
[0091] In one implementation, the target historical conversation can be obtained by simulating user actions, for example, by using an automated script or browser plugin to retrieve the currently visible text content from the display area of the conversation page.
[0092] As another implementation method, the requests sent by the user to the large language model and the responses returned by the large language model can be captured by intercepting or monitoring the data transmission between the client and the server, and these interaction records can be organized in chronological order to form the target historical dialogue.
[0093] After obtaining the target's historical dialogue, features can be extracted from the target's historical dialogue.
[0094] For example, word frequency analysis can be performed on historical dialogue texts to identify frequently occurring keywords. These keywords can then be used as thematic features of the dialogue. For instance, if words such as "nuclear energy" and "weapons" appear frequently in historical dialogues, they can be identified as sensitive thematic features.
[0095] As another example, it is also possible to identify whether there are specific sentence structures or phrases in historical conversations, such as "how to make it" or "tell me the method," through preset regular expressions or pattern matching rules. These patterns can serve as features of potential attack intentions.
[0096] Step 22: Based on the features of historical dialogues, generate adversarial test texts to test the target historical dialogues.
[0097] Here, since the adversarial test text is generated based on the features of historical dialogue, the adversarial test text can combine the specific context of the target historical dialogue, rather than using fixed test content that is independent of the context, so that the test content is targeted to the current session page.
[0098] In one implementation, a series of adversarial text templates can be predefined, such as "Please explain in detail the [sensitive operation] steps of [topic]". Then, based on the extracted historical dialogue features, the topics or keywords from the historical dialogue features are filled into placeholders in these templates to generate specific adversarial test text. For example, if the historical dialogue features show that the user is interested in "chemicals" and has an intention to "manufacture", then the template can be filled to generate "Please explain in detail the manufacturing steps of chemicals".
[0099] As another approach, a rule-based text generator can be constructed. This generator selects appropriate words from a predefined attack vocabulary based on sensitive information or potential risk points identified in historical dialogue features, and combines them with simple grammatical rules to construct adversarial statements. For example, if historical dialogue features indicate that the model is open to a specific topic, a leading question directly targeting that topic can be generated.
[0100] Step 23: Enter adversarial test text in the conversation page and obtain the test results of the target large language model's response to the adversarial test text.
[0101] The adversarial test text is input into the session page and sent to the target large language model. The target large language model responds to the adversarial test text, and the site obtains the response and uses it as the test result, or further processes the response to obtain the test result.
[0102] In one implementation, user input behavior on the conversation page can be simulated using automated tools. For example, keyboard input events can be simulated to input the generated adversarial test text word by word or all at once into the dialogue input box, and the "send" button can be simulated to submit the text to the large language model.
[0103] As an alternative implementation, the API interface for interaction between the session page backend and the large language model can be directly called, sending the adversarial test text as a request parameter. After the text is entered and submitted, the response from the large language model can be obtained.
[0104] For example, by parsing the DOM structure of the session page, the response text displayed on the page by the large language model can be identified and extracted. For instance, specific speech bubbles or text areas can be located to extract the model's output. Furthermore, by listening to network requests, the API response data returned by the large language model can be captured, and the model's response text can be parsed from it as test results.
[0105] In the aforementioned testing method for large language models, the target historical dialogue is obtained from the conversation page, and its features are extracted. Then, adversarial test text is generated based on these features and input into the conversation page to obtain test results. This achieves automated testing of the target large language model in conjunction with the dialogue context. Compared to the method of manually pre-writing fixed test samples, this embodiment automatically generates test content, eliminating the need for manual writing. Furthermore, the test content is dynamically generated based on the current dialogue context, thus solving the technical problems of insufficient test coverage, reliance on manpower, and low efficiency in the background technology. This achieves the beneficial effects of improving test coverage and testing efficiency.
[0106] In one embodiment, the target historical dialogue includes a multi-turn historical dialogue in the session page. After obtaining the test results of the target large language model's response to the adversarial test text, the method may further include step 24, updating the multi-turn historical dialogue in the session page based on the test results to obtain the updated multi-turn historical dialogue in the session page.
[0107] Specifically, please refer to Figure 3 Step 24 may include steps 241 to 243, wherein:
[0108] Step 241: Update the test results to the target historical dialogue in the conversation page to obtain the updated target historical dialogue.
[0109] The target historical dialogue can include a series of interactions between the user and the large language model, such as the user asking questions, the model providing answers, and the user continuing to ask questions or follow up based on the answers.
[0110] After obtaining the test results, the adversarial test text and the test results of the target large language model's response to the adversarial test text can be added to the target's historical dialogue, so that the target's historical dialogue grows as the test progresses.
[0111] It should be noted that the multi-turn dialogue structure can more realistically simulate the actual interaction process between users and large language models, providing rich contextual information. For example, the target historical dialogue can be a text sequence containing alternating user questions and model answers, or a structured dialogue tree that records the dialogue paths of different branches.
[0112] Furthermore, the test results can be updated by adding the large language model's response to the adversarial test text as a new dialogue turn to the current target historical dialogue. For example, the model's response text can be directly appended to the end of the historical dialogue record, or the response content can be structurally integrated with the historical dialogue, such as storing each round of questions and answers as key-value pairs. This ensures that subsequent tests can be conducted based on the latest dialogue context, thereby simulating a continuous interaction process and enabling the test to capture the model's behavioral changes during the dialogue evolution.
[0113] Step 242: Based on the updated target historical dialogue, extract new historical dialogue features and generate new adversarial test text, and input the new adversarial test text into the session page to obtain new test results.
[0114] After the target historical dialogue is updated, the entire dialogue sequence can be re-analyzed to capture the latest dialogue state and contextual information. For example, the updated dialogue can be semantically encoded, its state tracked, and key information extracted to form new historical dialogue features.
[0115] Based on these new features, adversarial test texts that are more relevant and targeted to the current dialogue context can be generated using mechanisms such as policy networks. For example, an encoder based on the Transformer architecture can be used to semantically encode the updated dialogue, or rule-based or machine learning methods can be used to track the dialogue state in order to identify key information such as user intent and dialogue topics.
[0116] Furthermore, the newly generated adversarial test text can be sent to the conversation page as user input, prompting the target large language model to generate new responses. This simulates continuous user interaction with the model in multi-turn dialogues and allows observation of the model's behavior in different contexts. For example, the new adversarial test text can be sent to the model as input parameters by calling the large language model's API, and the model's response can be received as the new test result.
[0117] Step 243: Repeat the above update and generation process to conduct multi-turn dialogue tests on the target large language model under the conversation page to obtain multi-turn historical dialogues.
[0118] In some implementations, the above steps can be continuously repeated, enabling the testing process of the large language model to automatically conduct multi-turn dialogues. This constructs a complete and in-depth multi-turn historical dialogue, which is used to comprehensively evaluate the performance and potential defects of the large language model in complex, continuous interaction scenarios. For example, a maximum number of dialogue turns or a test objective condition can be set, and the process can be stopped when the condition is met.
[0119] This application's solution addresses the challenge of traditional testing methods comprehensively evaluating model performance in complex dialogue scenarios by extending the testing of large language models from single interactions to continuous multi-turn dialogues. Specifically, it involves acquiring the target historical dialogue from the conversation page, extracting its historical dialogue features, generating adversarial test text based on these features, and inputting it into the conversation page to obtain test results. Furthermore, the target historical dialogue is defined as a sequence containing multiple turns of interaction. When the model responds to the adversarial test text, the test result is immediately updated in the current target historical dialogue, resulting in an updated target historical dialogue containing the latest interaction content. Subsequently, new historical dialogue features can be extracted based on this updated dialogue, leading to a more comprehensive and real-time understanding of the dialogue context. Based on these new and richer historical dialogue features, new, more targeted adversarial test texts highly relevant to the current dialogue state can be generated. Furthermore, the system automatically performs continuous multi-turn dialogue tests on the target large language model within the conversation page. Through iterative and adaptive testing mechanisms, the testing process can simulate continuous interactions between real users and the model, thereby constructing multi-turn historical dialogues with depth and breadth. This not only reveals deep-seated problems that are difficult to detect in single-turn interactions, such as contextual comprehension errors, memory loss, logical inconsistencies, or leakage of sensitive information, but also effectively evaluates the robustness and security of the model in long-term, complex dialogue scenarios, significantly improving the comprehensiveness and effectiveness of the test. Moreover, it can assess the robustness and stability of the model when facing continuous adversarial or complex situations.
[0120] Compared to methods that only conduct single-turn or pre-set dialogue tests, this embodiment can construct more representative and challenging multi-turn historical dialogues, thereby more accurately revealing the potential risks and behavioral patterns of large language models in real-world application scenarios, and improving the effectiveness and coverage of the test.
[0121] In one embodiment, the session page includes multiple dialog windows, and each dialog window includes a multi-turn history of the conversation. Please refer to [reference needed]. Figure 4Step 21, obtaining the target historical conversation from the conversation page, can specifically include step 211, where:
[0122] Step 211: Obtain the multi-turn historical dialogues from multiple dialogue windows to obtain the target historical dialogue.
[0123] In this context, a dialog window can be understood as a logical or physical partition within the same session page, used to carry independent conversation flows. For example, a dialog window can appear as different tabs in a browser, an independent chat panel within an application, or multiple dialog boxes displayed side-by-side on the same interface.
[0124] Each dialog window independently maintains its interaction history with the large language model, recording a series of interactions between the user and the large language model in each independent dialog window.
[0125] In situations with multiple dialogue contexts, instead of retrieving historical dialogues from a single source, the system proactively identifies and collects historical dialogues from all relevant dialogue windows. Specifically, it can traverse all available dialogue windows on the session page, extract complete or specified-range multi-turn historical dialogues within each window, and integrate these independent dialogue histories or use them as independent test units to form a comprehensive target historical dialogue for subsequent analysis.
[0126] In this embodiment, by explicitly stating that multiple dialogue windows exist within the conversation page, and that each dialogue window contains independent multi-turn historical dialogues, the method for obtaining the target historical dialogue is refined. In the basic scheme, obtaining the target historical dialogue in the conversation page is the foundation for subsequent extraction of historical dialogue features and generation of adversarial test text. When multiple independent dialogue contexts exist within the conversation page, obtaining the historical dialogue without differentiation may result in incomplete or inaccurate context information.
[0127] It should be noted that by actively acquiring multi-turn historical dialogues from multiple dialogue windows, the obtained target historical dialogues are ensured to comprehensively reflect the interaction history of the large language model in different dialogue contexts. This results in richer and more accurate extracted historical dialogue features, such as the ability to capture the model's behavioral patterns under different dialogue topics or user intentions. Based on more accurate historical dialogue features, the generated adversarial test texts will be more targeted and effective, and will be able to more comprehensively detect potential vulnerabilities and robustness issues of the large language model in multi-context environments.
[0128] Through the above technical solution, this application can effectively solve the problem of incomplete or inaccurate acquisition of historical dialogues when multiple independent dialogue contexts exist on a conversation page. By explicitly acquiring multi-turn historical dialogues from multiple dialogue windows, the obtained target historical dialogues can more comprehensively and accurately reflect the interaction history of the large language model in different contexts, making subsequent historical dialogue feature extraction more accurate. The generated adversarial test texts can better simulate multi-threaded and multi-task dialogue scenarios in the real world, thereby significantly improving the coverage and effectiveness of large language model testing and ensuring a more comprehensive and in-depth evaluation of the model's robustness and security.
[0129] In one embodiment, please refer to Figure 4 Step 21, extracting the historical dialogue features of the target historical dialogue, can specifically include steps 212 to 215, wherein:
[0130] Step 212: Semantically encode the target historical dialogue to obtain the semantic representation of the target historical dialogue.
[0131] Semantic representation refers to the representational information that characterizes the semantic content of a target historical dialogue after semantic encoding at the target level. For example, semantic representation can be the conversion of the original textual dialogue content into a numerical representation that can be understood and processed by a machine.
[0132] As an example, pre-trained language models (such as BERT, GPT series, RoBERTa, etc.) can be used to encode dialogue text to obtain its word vectors or sentence vectors; alternatively, sequence models such as recurrent neural networks (RNN) or long short-term memory networks (LSTM) can be used to process the text to extract its semantic features.
[0133] It should be noted that semantic representation is a numerical vector or matrix obtained after semantic encoding. It carries the semantic information of the dialogue text and can serve as the basis for subsequent processing. For example, semantic representation can be represented as a fixed-dimensional vector, such as sentence embedding, or as a serialized vector, such as a sequence of embedding vectors for each word or subword.
[0134] Step 213: Based on semantic representation, perform dialogue state tracking on the target historical dialogue to obtain the dialogue state.
[0135] The dialogue state is used to represent the contextual information of the target's historical dialogue.
[0136] It should be noted that tracking the dialogue state of a target's historical dialogue can involve tracking and analyzing the context of the target's historical dialogue in order to dynamically identify and maintain key information during the dialogue process, such as user goals, slot values, or dialogue history.
[0137] As an example, dialogue state tracking can be achieved by establishing a series of rules to identify and update the dialogue state. Alternatively, machine learning methods can be used to train models (such as RNNs or Transformers) to learn the dialogue state update logic from semantic representations; or, predefined domain knowledge structures (ontologies) can be utilized to track and manage the dialogue state.
[0138] For example, the dialogue state is the result of dialogue state tracking, which represents the context information of the current dialogue in a structured way. It can be represented as key-value pairs (e.g., {"User Intent": "Query Weather", "City": "Beijing"}), or as a representation encoded as a continuous numerical vector, or as structured data containing multiple slots and their corresponding values (e.g., a JSON object).
[0139] Step 214: Extract key information based on semantic representation.
[0140] The key information includes at least one of the following: user intent, dialogue topic, and historical adversarial test patterns.
[0141] Here, user intent can be used to characterize the purpose to be achieved in the current dialogue; dialogue topic can be used to characterize the core content of the dialogue; and historical adversarial test patterns can be used to characterize the adversarial test methods that have appeared in the target's historical dialogues.
[0142] As an example, user intent can be determined by identifying the user's main purpose in the dialogue from the semantic representation through an intent recognition model (e.g., a classifier-based or sequence labeling model); the dialogue topic can be determined by identifying the core issues of the dialogue from the semantic representation through a topic model (e.g., LDA, NMF) or a clustering algorithm; and the historical adversarial test patterns can be determined by analyzing past adversarial test records to identify common patterns that lead to model failure or poor performance (e.g., specific types of questions, leading statements, logical traps, etc.).
[0143] Step 215: Integrate semantic representation, dialogue state, and key information to obtain historical dialogue features.
[0144] Here, historical dialogue features can be achieved by directly concatenating vectors of different features into a longer vector; attention mechanisms can be used to weight and fuse different features to highlight important information; or, specialized network structures (such as gating mechanisms) can be designed to fuse different types of information.
[0145] After acquiring the target historical dialogue from the conversation page, to extract historical dialogue features more comprehensively and accurately to guide the generation of subsequent adversarial test texts, the target historical dialogue is first semantically encoded, transforming it into a machine-understandable semantic representation. This semantic representation serves as the foundation, used for two purposes: firstly, to track the dialogue state, dynamically capturing and maintaining the contextual information of the dialogue to form a dialogue state; secondly, to extract key information from the dialogue, such as user intent, dialogue topics, and past adversarial test patterns. Finally, these semantic representations, dialogue states, and key information are integrated to form a comprehensive and structured historical dialogue feature. Through multi-dimensional and deep feature extraction, the subsequently generated adversarial test texts can more accurately simulate the complexity and potential challenges of real-world dialogue scenarios, effectively testing the weaknesses of large language models in understanding, reasoning, or responding to specific situations. This ensures that the adversarial test texts can accurately test the specific capabilities, contextual understanding, or potential vulnerabilities of the target large language model, significantly improving the effectiveness and relevance of the testing.
[0146] In one embodiment, please refer to Figure 5 Step 22, generating adversarial test text based on historical dialogue features to test the target historical dialogue, may specifically include steps 221 to 223, wherein:
[0147] Step 221: Convert the historical dialogue features into state representations.
[0148] In this context, the state representation refers to the input representation used by the policy network to make decisions.
[0149] Specifically, the state representation here has a different meaning from the dialogue state in step 213. The dialogue state is used to represent the contextual information of the target's historical dialogue and belongs to the understanding result of the dialogue itself; while the state representation is used as the basis for the policy network to make decisions and belongs to the input information for decision-making.
[0150] Here, historical dialogue features are the original set of information describing the current dialogue context. In order for historical dialogue features to be effectively processed by reinforcement learning models, they need to be transformed into a structured and numerical representation, namely, state representation.
[0151] Specifically, semantic representations, dialogue states, and key information from historical dialogue features can be concatenated, encoded, or dimensionality-reduced to form a fixed-dimensional vector as the state representation. Furthermore, deep learning models, such as recurrent neural networks or Transformer encoders, can be used to further extract and fuse features from historical dialogue features, outputting a vector that captures the deep semantics and contextual information of the dialogue as the state representation.
[0152] Step 222: Input the state representation into the reinforcement learning-based policy network, and output the adversarial test action corresponding to the state representation based on the policy network.
[0153] Among them, the policy network refers to a network based on reinforcement learning that outputs adversarial test actions according to the state representation. The policy network can be used to output adversarial test actions corresponding to the input state representation.
[0154] It should be noted that after receiving the state representation of the current environment, the policy network can output adversarial test actions based on the policies it has learned. In this context, adversarial test actions are "adversarial test strategies" used to generate adversarial test text. Adversarial test actions can be used to represent the strategy that is most likely to lead to a valid test result in the current dialogue state.
[0155] As an example, a policy network can be a deep neural network, such as a multilayer perceptron, recurrent neural network, or Transformer model, which optimizes its internal parameters by learning from a large amount of interaction data to select the optimal action given a state. Alternatively, a policy network can employ an Actor-Critic architecture, where the Actor network outputs the action, and the Critic network evaluates the value of that action. Through collaborative training, the policy network can learn more stably and efficiently.
[0156] Step 223: Convert the adversarial test actions into text to obtain the adversarial test text.
[0157] Among them, adversarial test text can be human-readable and understandable natural language text converted from adversarial test actions represented by abstract, discrete or continuous numerical values.
[0158] Specifically, if the adversarial test actions are discrete action spaces, such as selecting a predefined attack template, modifying a keyword, or inserting a phrase, an action ID can be mapped to a specific text fragment or text generation instruction through a mapping table or rule engine. Alternatively, if the adversarial test actions are continuous vector representations, they can be used as conditional input to a text generation model, such as the GPT series, BART, or T5 model, which then generates the corresponding adversarial test text based on the action vectors.
[0159] This solution optimizes the process of generating adversarial test text based on historical dialogue features by introducing a reinforcement learning mechanism. First, the historical dialogue features extracted from the target historical dialogue obtained from the conversation page are transformed into a state representation understandable by the policy network, ensuring that the policy network can fully perceive the current dialogue context. Then, this state representation is input into the reinforcement learning-based policy network. Based on its pre-learned policy and the current state representation, it intelligently outputs an adversarial test action. This adversarial test action is further converted into specific natural language text, resulting in the final adversarial test text. Based on this, adaptive and intelligent generation of adversarial test text can be achieved, allowing the testing process to dynamically adjust the testing strategy according to the historical dialogue context. This effectively solves the problem of traditional methods lacking intelligence and adaptability in generating adversarial test text. It not only improves the efficiency of adversarial test text generation but also generates more targeted and effective test text, thereby more comprehensively and deeply uncovering potential vulnerabilities, biases, or inappropriate responses in the target large language model, improving the coverage and accuracy of large language model testing.
[0160] In one embodiment, the step of inputting a state representation into a reinforcement learning-based policy network and outputting an adversarial test action corresponding to the state representation based on the policy network includes processing steps one through four, wherein:
[0161] The first processing step involves generating multiple candidate adversarial test actions based on the policy network and the state representation.
[0162] After receiving the current state representation, the policy network no longer simply outputs a definite action, but explores and provides a series of possible action options.
[0163] For example, a policy network can be designed to output a probability distribution of an action and then sample multiple times from that distribution to obtain multiple candidate actions; or, a policy network can also include an exploration mechanism, such as noise injection or multi-head output, to generate different action proposals given the same state representation.
[0164] The second step involves conducting a feasibility assessment of multiple candidate adversarial test actions to obtain the predicted success rate of each candidate adversarial test action.
[0165] Among them, the success rate prediction value can be used to predict the probability that each generated candidate action will successfully reveal potential problems of the large language model in actual testing.
[0166] As an example, each candidate action can be evaluated using a separate predictive model, which can be a trained machine learning model that takes the features of the candidate action and the current state representation as input and outputs a probability value between 0 and 1, representing the expected success rate of the action.
[0167] As another example, the success rate of each candidate action can be estimated based on predefined rules or heuristics to obtain the corresponding success rate prediction value.
[0168] In the third step of the process, when the predicted success rate is not lower than the feasibility threshold, the candidate adversarial test action with the highest value is selected to obtain the adversarial test action.
[0169] When at least one candidate action is considered to have sufficient potential for success, the actions that are expected to bring the greatest benefit or the most important test results can be prioritized.
[0170] Here, value can be estimated by another value function or value network. Value can be used to quantify the cumulative reward or importance that may be obtained after performing an action.
[0171] Understandably, the feasibility threshold is a pre-set benchmark used to determine whether an action has a sufficient success rate.
[0172] In the fourth processing step, when the predicted success rate is lower than the feasibility threshold, an adversarial test action is randomly selected from multiple candidate adversarial test actions to obtain the adversarial test action.
[0173] When the success rate of all candidate actions is lower than the preset minimum requirement, an exploratory strategy can be adopted, which is to randomly select an action. This helps to break out of the current predicament, try new and unverified test paths, thereby avoiding getting stuck in local optima and increasing the chance of discovering new vulnerabilities.
[0174] Through the above technical solution, this application can effectively solve the problems of blindness and uncertainty that may exist when policy networks directly output actions. By generating multiple candidate adversarial test actions and evaluating their feasibility, test actions can be selected more intelligently, avoiding ineffective attempts on low-success-rate actions, thereby significantly improving the efficiency and targeting of testing. In addition, when high-success-rate actions are lacking, the random selection mechanism ensures effective exploration of the unknown test space, enhances the robustness of testing, enables large language models to be evaluated more comprehensively, and helps to discover potential vulnerabilities and security risks that are difficult to detect by traditional methods, making the adversarial testing process more flexible and efficient.
[0175] In one embodiment, the policy network generation step includes training steps one through three, wherein:
[0176] The first step in training is to obtain training data consisting of multiple test trajectories.
[0177] The test trajectory includes the state representation at each time step, adversarial test actions, and corresponding rewards.
[0178] Here, training data can serve as the foundation for reinforcement learning policy networks to learn.
[0179] It can be explained that training data can be a series of interaction sequences that record the actions taken in different states and the immediate feedback brought about by these actions.
[0180] For example, training data can come from actual interactive tests on the target large language model, or it can be generated through simulated environments or expert experience. For instance, training data can be collected by recording the dialogue between human testers and the large language model, or by automating interactions with the model through pre-set test scripts.
[0181] It should be noted that the test trajectory can completely record the interaction process from a certain initial state through a series of time steps. Each time step includes the state representation of the current environment, the adversarial test action selected by the policy network in that state, and the reward given by the environment after executing the action. The state representation is an abstraction of the current dialogue context, the adversarial test action is the instruction output by the policy network, and the reward is a quantitative evaluation of the effect of the action.
[0182] The second training step involves calculating the reward at each time step based on the reward function.
[0183] The reward function is determined based on at least one of the following: the degree of matching between the test content and the test objective, the degree of leakage of sensitive information, and the model response length.
[0184] Here, the reward function is a key component of the guidance policy network learning in reinforcement learning. The reward function is used to define the "good or bad" of performing a specific action in a given state.
[0185] It should be noted that the reward function can quantify the effectiveness of each adversarial test action. The reward can be calculated based on preset rules or predicted using a machine learning model.
[0186] Specifically, the matching degree between test content and test objective measures whether the generated adversarial test text effectively targets the pre-set test objective, such as whether it successfully induces the model to produce a specific type of erroneous response. The degree of sensitive information leakage assesses whether the model discloses information that should not be made public when responding to adversarial test text. Model response length can serve as an indicator of the quality or complexity of the model's response.
[0187] In particular, the above indicators can be used individually or in combination to form a comprehensive reward signal.
[0188] In the third training step, the parameters of the policy network are updated with the optimization objective of maximizing the cumulative reward of the test trajectory, resulting in the updated policy network.
[0189] The goal of reinforcement learning with policy networks is to find an optimal policy through trial and error, maximizing the cumulative reward gained over long-term interactions. Therefore, updating the parameters of the policy network is a crucial step in achieving this goal. Optimization algorithms (such as gradient descent) are used to adjust the weights and biases within the network, making the policy network more likely to choose actions that yield high rewards in a given state.
[0190] This application addresses the problem of policy networks effectively generating adversarial test actions by constructing a reinforcement learning-based training framework. Specifically, firstly, training data consisting of multiple test trajectories is acquired. These trajectories record in detail the adversarial test actions taken by the policy network under different dialogue states, along with the rewards obtained for these actions. This training data provides rich experience for the policy network's learning. Secondly, based on a pre-defined reward function, the reward at each time step can be accurately calculated. This reward function comprehensively considers key indicators such as the matching degree between the test content and the test target, the degree of sensitive information leakage, and the model response length, thereby ensuring that the reward signal accurately reflects the effectiveness of the adversarial test actions. Finally, with the optimization objective of maximizing the cumulative reward of the test trajectories, the parameters of the policy network are continuously updated using a reinforcement learning algorithm. Based on this, the policy network can gradually learn strategies for generating optimal adversarial test actions under different states, enabling it to more effectively generate adversarial test text targeting the target large language model in subsequent test executions.
[0191] The aforementioned technical solution effectively trains and optimizes the policy network used to generate adversarial testing actions. By introducing training data based on test trajectories, a finely designed reward function, and an optimization objective that maximizes cumulative rewards, the policy network can learn more efficient and targeted adversarial testing strategies. This enables the generated adversarial test text to more accurately detect potential vulnerabilities and weaknesses in the target large language model, such as more effectively identifying issues like sensitive information leakage, inappropriate content generation, or logical fallacies. Therefore, it significantly improves the automation and effectiveness of large language model testing, reduces the cost and complexity of manual testing, and ensures the comprehensiveness and depth of the testing.
[0192] In one embodiment, please refer to Figure 6Step 23, entering adversarial test text in the conversation page and obtaining the test results of the target large language model's response to the adversarial test text, can specifically include steps 231 to 233, wherein:
[0193] Step 231: Concatenate the adversarial test text with the target historical dialogue to construct the first dialogue context.
[0194] Here, the first dialogue context can be dialogue context information that is spliced together by integrating the generated adversarial test text into the existing dialogue history in order to make the target historical dialogue look more natural and more in line with the actual dialogue scenario. This can improve the concealment and effectiveness of adversarial testing.
[0195] As an example, adversarial test text could be simply appended to the end of the target historical dialogue to form a continuous dialogue flow; or, adversarial test text could be inserted into a specific position in the target historical dialogue, such as after a relevant question, based on the logic or semantics of the dialogue, to better simulate real user interaction patterns.
[0196] Step 232: Apply adversarial perturbation to the adversarial test text in the first dialogue context to obtain the second dialogue context.
[0197] Here, the second dialogue context can be dialogue context information that applies adversarial perturbations to the adversarial test text in the first dialogue context in order to further enhance the adversarial nature of the adversarial test text, making it more difficult for the target large language model to recognize or avoid it. This can more effectively detect the robustness and security of the model.
[0198] As an example, one could introduce subtle, imperceptible character-level or word-level modifications into adversarial test text, such as synonym substitution, character addition or deletion, or spelling errors, to create adversarial perturbations. The aim is to change the model’s internal representation without altering human understanding of the text.
[0199] As another example, gradient-based attack methods can be used to add perturbations to the adversarial test text that maximize the model’s error response, based on the gradient information of the target large language model.
[0200] Step 233: Input the second dialogue context into the conversation page and obtain the test results of the target large language model's response to the adversarial test text.
[0201] The performance of the target large language model is evaluated by submitting carefully constructed and perturbed dialogue contexts to obtain its responses to more complex and challenging inputs.
[0202] For example, the second dialogue context can be sent to the target large language model via an API interface, and the response returned by the model can be received as the test result; or, the second dialogue context can be entered word by word or in whole paragraphs into the input box of the conversation page by simulating the user's input behavior on the conversation page, and the response displayed by the model on the page can be captured as the test result.
[0203] Through the above technical solution, this application can effectively solve the problem that directly inputting adversarial test text may not fully stimulate the potential vulnerabilities of the target large language model. By concatenating the adversarial test text with the target's historical dialogue and applying adversarial perturbations, the adversarial input is made more covert and context-dependent, thereby enabling a deeper exploration of the target large language model's response to complex and subtle adversarial inputs, improving the effectiveness and depth of the test, and thus more accurately evaluating the model's robustness and security.
[0204] In some of the embodiments described above in this application, a method is proposed to generate adversarial test text and input it into a session page to obtain test results. However, in actual testing of a target large language model, especially in large-scale or high-concurrency testing scenarios, sending all test requests indiscriminately may cause the target large language model to refuse to respond due to exceeding its preset request processing capacity or API call limits, or may affect the stability and efficiency of the test due to excessive request processing latency, or even cause the test to be interrupted, thus failing to effectively obtain complete test results.
[0205] Based on this, in one embodiment, if there are multiple test requests, before entering adversarial test text in the session page, step B22 may be included to filter the multiple test requests, obtain multiple target test requests, and determine the test strategies corresponding to the multiple target test requests.
[0206] Please refer to Figure 7 Step B22 may specifically include steps B221 to B223, wherein:
[0207] Step B221: Based on the request constraints of the target large language model, filter multiple test requests to obtain the target test request.
[0208] The request constraints include at least one of the following: rate limit, single-term term limit, and number of concurrent connections.
[0209] Among them, request constraints refer to the restrictive conditions set by the target large language model when processing external requests, which aim to maintain its service stability, fair allocation of resources and prevent abuse.
[0210] In this embodiment, these request constraints may include, but are not limited to: rate limiting, which restricts the number of requests allowed per unit time; single-transaction token limit, which restricts the maximum number of tokens allowed in a single request to control processing complexity and resource consumption; and concurrent connection limit, which restricts the number of requests processed simultaneously. Additionally, they may include request body size limits, API key usage frequency limits, etc.
[0211] For example, filtering multiple test requests refers to evaluating and selecting all test requests to be sent based on preset rules or strategies. This ensures that only requests that meet the constraints of the target large language model are sent, thereby avoiding request failures or service interruptions due to violations. For instance, initial filtering can be performed based on factors such as request priority, request content size, and expected response time; alternatively, dynamic adjustments can be made based on the current load of the target large language model and known API limitations.
[0212] Step B222: When the response time of the target test request exceeds the preset timeout threshold, the target test request is determined to have timed out.
[0213] The preset timeout threshold refers to the maximum allowed time for the target large language model to respond to a test request. Once the actual response time exceeds this threshold, the request will be judged as timeout.
[0214] It can be noted that the timeout threshold can be set empirically based on the performance characteristics of the target large language model, network latency, and the tolerance of the test. For example, it can be set to 5 seconds, 10 seconds, or longer.
[0215] After sending the target test request, a timer is started. When the timer reaches a preset timeout threshold, it checks whether a response has been received from the target large language model. If no response is received, the request is considered unsuccessful, i.e., a timeout has occurred. This ensures that the testing process does not wait indefinitely for unresponsive requests, thereby improving testing efficiency.
[0216] Step B223: If multiple consecutive target test requests time out, reduce the sending rate of the target test requests.
[0217] If multiple target test requests sent within a short period of time fail to receive timely responses, it may indicate that the target large language model is facing issues such as high load, network failure, or service anomalies. For example, this could be set to indicate that M out of the most recent N requests timed out, or that the proportion of timed-out requests exceeds a certain percentage within a certain time window.
[0218] Upon detecting anomalies such as consecutive timeouts, the frequency of sending test requests to the target large language model is proactively reduced to alleviate its pressure and provide a buffer time for it to recover normal service, while avoiding exacerbating the problem due to continuous request sending. For example, an exponential backoff strategy can be adopted to halve the sending rate; or the sending interval can be dynamically adjusted based on the number and frequency of timeout requests.
[0219] This application's solution introduces a request management mechanism to intelligently process test requests before inputting adversarial test text into the conversation page. Specifically, when multiple test requests are pending, these requests are filtered based on the request constraints set by the target large language model, such as rate limits, single-word limit, and concurrent connection count. This ensures that only requests that meet or are adjusted to meet these constraints are further processed, thus generating target test requests. Subsequently, these target test requests are sent to the conversation page. During the sending process, the response time of each target test request can be continuously monitored. Once the response time of a target test request exceeds a preset timeout threshold, the request is determined to have timed out. When multiple consecutive target test requests time out are detected, this usually indicates that the target large language model may be under high load or instability. In this case, the sending rate of target test requests can be proactively reduced, effectively avoiding further burdening the target large language model due to continuous high-frequency request sending. This ensures the stability and continuity of the testing process, enabling effective multi-turn dialogue testing of the target large language model even in complex or high-pressure environments.
[0220] Through the above technical solution, this application effectively solves the problems of test interruption, low efficiency, or unreliable results caused by excessive request volume or abnormal model response during large-scale or high-concurrency testing of target large language models. By intelligently filtering and rate controlling test requests, it ensures that the testing process always meets the request constraints of the target large language model, avoiding request failures due to violations. Simultaneously, the introduced timeout detection and dynamic rate adjustment mechanism enables timely detection and response to abnormal responses or overload situations of the target large language model, effectively protecting the target large language model and ensuring the stability and continuity of the testing task. This makes the testing process more robust and efficient, stably acquiring the response of the target large language model under various adversarial testing scenarios, thereby improving the accuracy and reliability of the test results.
[0221] In one embodiment, after obtaining the test results of the target large language model's response to the adversarial test text, the method may further include step 25, generating empirical data based on the test results, and iteratively optimizing the policy network that generates adversarial test actions based on the empirical data.
[0222] Please refer to Figure 8 Step 25 may specifically include steps 251 to 254, wherein:
[0223] Step 251: Calculate the success rate of the test based on the test results.
[0224] Among these metrics, test success rate can be used to measure the extent to which adversarial test texts achieve the preset test objectives. For example, whether it successfully induces a large language model to produce an undesirable response (such as generating harmful content, refusing to answer, or generating hallucinations).
[0225] As an example, the success rate of the test can be calculated based on whether the target large language model's response to the adversarial test text triggers a preset failure condition. If the failure condition is triggered, the success rate can be set to a high value; otherwise, it is set to a low value.
[0226] As another example, a comprehensive score can be obtained based on multiple dimensions such as the degree of deviation between the model response and the expected response, the degree to which the model response contains sensitive information, or the length of the model response, to obtain a quantitative success value.
[0227] Step 252: Evaluate the effectiveness of the adversarial test text based on the test success rate.
[0228] Among them, the test results can be a comprehensive evaluation of the adversarial test text in discovering potential problems in the target large language model.
[0229] As one approach, the evaluation of test effectiveness can be achieved by directly using the calculated test success rate as the test result, or by combining factors such as test success rate, computational resource consumption in generating adversarial test texts, and test time, and using a weighted average or machine learning model for comprehensive evaluation to provide a more comprehensive feedback on test quality.
[0230] Step 253: Based on the test results, generate and store the experience data to represent the test process.
[0231] Among them, the empirical data is a set of data that records key information of this test process, which can provide training samples or feedback signals for subsequent optimization of the generation of adversarial test actions.
[0232] Here, empirical data can include the state representations used in this test, the generated adversarial test actions, and the evaluated test results. Furthermore, empirical data can also include auxiliary information such as the target large language model's raw response, test environment parameters, and timestamps to provide a more comprehensive context.
[0233] Step 254: Based on empirical data, optimize the policy network that determines the adversarial test actions corresponding to the adversarial test text according to the characteristics of historical dialogues.
[0234] Here, by leveraging historical testing experience to improve the generation strategy for future adversarial testing actions, the testing process is automated and continuously optimized, thereby improving testing efficiency and effectiveness.
[0235] For example, empirical data can be used as training samples for reinforcement learning to update the parameters of a policy network, enabling it to generate more effective adversarial test actions. Alternatively, empirical data can be used to build a predictive model that can predict the potential test effects of different adversarial test actions in specific states, thereby guiding action selection.
[0236] This application's solution constructs a closed-loop test optimization process by introducing mechanisms for calculating test success rate, evaluating test effectiveness, and generating and storing experience data after obtaining test results. When the target large language model responds to adversarial test texts and generates test results, it no longer stops at simply obtaining the results but further quantifies (calculates test success rate) and evaluates (assesses test effectiveness) the effectiveness of the test. If a certain adversarial test action produces high test effectiveness in a specific context, it will tend to select or generate similar adversarial test actions again when encountering similar situations in the future. Conversely, if the test effectiveness is poor, it will learn to avoid the action or explore other alternatives. This feedback-based continuous learning and optimization mechanism ensures that the generation of adversarial test texts is no longer a blind attempt but can be intelligently adjusted and improved based on historical experience. This significantly improves testing efficiency and the ability to discover model vulnerabilities, enhances the efficiency and accuracy of discovering potential vulnerabilities in the target large language model, avoids repeated and ineffective tests, and ultimately improves the overall intelligence level and coverage of large language model testing.
[0237] In one embodiment, the empirical data includes at least the state representations corresponding to historical dialogue features, adversarial test actions, and test results.
[0238] It should be noted that experience data refers to the collection of information collected during the testing process for learning and optimization. It records the results observed after taking a certain action in a specific state and can be stored in a database, memory buffer, or file.
[0239] The state representation can be an abstract description of the current testing environment or dialogue context, and can be a vector, tensor or other data structure used to capture the semantics, dialogue state, key information, etc. of the target's historical dialogue.
[0240] The construction of state representations can include, but is not limited to: encoding historical dialogue text through a pre-trained language model, or extracting structured information such as dialogue intent and slot values through a dialogue state tracker.
[0241] In addition, adversarial test actions can be specific operations that are selected to be performed under a given state. The purpose is to generate adversarial test text, which can be a text generation instruction, a template filling operation, a parameter adjustment instruction, or a more complex strategy.
[0242] Finally, the test results can be a quantitative assessment of the quality or adversarial strength of the target large language model's response after the adversarial test actions are executed. This can be measured based on indicators such as test success rate, degree of sensitive information leakage, and unexpectedness of the model's response.
[0243] Step 254, optimizing the policy network based on empirical data to determine the adversarial test actions corresponding to the adversarial test text according to historical dialogue features, can specifically include steps 254a to 254c, wherein:
[0244] Step 254a: Determine the reward corresponding to the adversarial test action based on the test results.
[0245] In this context, the reward can be a signal used in reinforcement learning to evaluate the quality of an action. It is a numerical value that can be used to represent the immediate feedback obtained after performing an adversarial test action in a specific state.
[0246] One approach is to directly map test results to reward values; for example, the better the test results, the higher the reward value. Alternatively, rewards can be calculated comprehensively based on a preset reward function, taking into account multiple dimensions such as test success rate, model response safety, and model response length. In this way, abstract test results can be transformed into quantifiable learning signals.
[0247] Step 254b: Update the value estimate of selecting adversarial test actions under the state representation based on the reward.
[0248] Among them, value estimation can be the expected value of the future cumulative reward that can be obtained by choosing a certain adversarial test action in a given state, which can be used to reflect the perception of the advantages and disadvantages of different actions.
[0249] As an example, value-based methods such as Q-learning and SARSA can be used to update the Q-value table or value function iteratively through the Bellman equation; alternatively, deep Q-networks in deep reinforcement learning can be used to approximate the value function.
[0250] By learning from historical experience, we can more accurately predict the potential benefits of taking different actions under different conditions, thereby guiding future action choices.
[0251] Step 254c: Select the adversarial test action with the highest value based on the value estimate with a first probability, and randomly select an adversarial test action with a second probability.
[0252] The first probability can be the probability of choosing what is considered the optimal adversarial test action based on currently known value estimates, and can be used to represent the strategy determined by "utilizing" learned knowledge.
[0253] The second probability can be the probability of randomly selecting an adversarial test action, without relying on value estimation, to represent the strategy of "exploring" the unknown action space.
[0254] It should be noted that in this embodiment, the first probability and the second probability can balance the exploration and exploitation issues in reinforcement learning. By conducting random exploration with a certain probability, new and potentially better adversarial test actions can be discovered, avoiding getting trapped in local optima; at the same time, by utilizing known optimal actions with a higher probability, testing efficiency can be guaranteed. The sum of the first probability and the second probability is usually 1, and can be dynamically adjusted according to the testing stage or specific needs.
[0255] This application's solution constructs an adaptive adversarial test action selection mechanism by structurally defining empirical data and introducing a reward mechanism. Specifically, after obtaining the response results of the target large language model to the adversarial test text, the test success rate is first calculated based on the response results, and the test effect of this adversarial test text is evaluated based on this. Subsequently, the state representations, adversarial test actions, and evaluated test effects involved in this test are stored as empirical data. To optimize the generation of subsequent adversarial test actions, the stored test effects can be converted into corresponding reward values. This reward value quantifies the effect of a specific adversarial test action in a specific state. Based on these reward values, the value estimates of selecting each adversarial test action under different state representations can be continuously updated to reflect the cumulative benefits that each action may bring in the long term. In the actual selection of adversarial test actions, a strategy balancing exploration and utilization is adopted. It prioritizes the adversarial test action with the highest current value estimate with the highest probability, that is, it utilizes the known optimal strategy to improve testing efficiency. At the same time, in order to avoid getting trapped in local optima and to discover new, potentially more effective adversarial strategies, it also randomly selects an adversarial test action with a second probability for exploration. This mechanism enables the testing process to continuously learn from historical experience and dynamically adjust its testing strategies, thereby continuously improving the efficiency and coverage of adversarial testing. In this way, the proposed solution effectively addresses the challenge of using empirical data to guide subsequent adversarial testing actions, balancing exploration and exploitation, thus making the testing process more intelligent and efficient.
[0256] Through the above technical solution, this application effectively addresses the problem of balancing the utilization of known effective strategies and the exploration of unknown potential strategies in adversarial testing. By structurally storing experiential data and determining rewards based on test results, it is possible to continuously learn and optimize the value estimation of adversarial testing actions. This exploration and utilization strategy based on value estimation and probability selection enables the testing process to efficiently utilize verified attack methods while proactively exploring new attack paths and potential vulnerabilities, thereby significantly improving the intelligence level, efficiency, and coverage of adversarial testing for large language models and effectively discovering deeper vulnerabilities in the model.
[0257] In one embodiment, Figure 9 This document provides an interactive flowchart for a testing method for a large language model. Figure 9 As shown, to more clearly illustrate the implementation process of this embodiment, the following detailed explanation of the complete execution flow of this method in a multi-turn dialogue test is provided with a specific example. In this example, the site interacts with the target large language model through a conversation page, and the test objective is to detect whether the target large language model will leak personal privacy information.
[0258] (a) Constructing a test corpus
[0259] A pre-built test corpus is used to store structured and unstructured test data. The test corpus includes at least one of the following: test templates, sensitive topics, jailbreak commands, and adversarial samples. This test corpus can be built through web scraping, self-generation, or deployment of open-source databases.
[0260] The test corpora are categorized according to the test objectives, such as personal information mining, illegal content generation, and dissemination of false information. Each test corpus has a metadata tag, which is used to characterize at least one of the following: attack strength, stealth, and applicable scenarios.
[0261] Furthermore, it supports semantic retrieval of the test corpus based on the current dialogue context to match the test corpus that best fits the current context. In this example, based on the test objective "leakage of personal privacy information," relevant test corpus under the category of "personal information mining" is retrieved from the test corpus for use by the subsequent attack strategy generation module.
[0262] (ii) Dialogue History Analysis Module
[0263] The site first inputs the target large language model to form the first round of dialogue, which serves as the target historical dialogue. Let the target historical dialogue be... ,in This represents user input. The target large language model is aimed at The site converts the target history conversation into a structured format and performs initial cleaning, removing irrelevant characters and formatting errors. Subsequently, the site performs the following processing:
[0264] 1. Semantic encoding
[0265] The target historical dialogue is encoded using a pre-trained Transformer encoder E to generate a semantic representation. ,in, d represents the encoding dimension. The encoding process includes: adding positional encoding to each token to introduce positional information; calculating the association between tokens via a self-attention mechanism; and extracting high-level semantic features via a feedforward neural network. The output of the Transformer encoder can be represented as:
[0266] ;
[0267] in, The multi-head self-attention mechanism is calculated as follows:
[0268] ;
[0269] ;
[0270] ;
[0271] in , , The projection matrix of query, key, and value. To output the projection matrix, The dimension of the key.
[0272] 2. Dialogue status tracking
[0273] The dialogue state is calculated using state tracker F. The dialogue state S represents the context information of the current dialogue. The state tracker uses a bidirectional LSTM network to capture the bidirectional dependencies of the dialogue history.
[0274] ;
[0275] ;
[0276] ;
[0277] in, and These are the final hidden states of the forward LSTM and the backward LSTM, respectively, and [;] represents the concatenation operation.
[0278] 3. Key Information Extraction
[0279] Extract key information from the target's historical dialogues, including user intent U (identifying the purpose of the current dialogue), dialogue topic T (determining the core content of the dialogue), and historical attack patterns M (identifying historically occurring attack methods). Here, , Each item in the table can be implemented by a corresponding classifier:
[0280] ;
[0281] ;
[0282] ;
[0283] in , , For weight parameters, , , This is a bias term.
[0284] semantic representation Dialogue status Key information , , Integrate and generate the feature vectors required by the system. The output historical feature representation is used by the attack strategy generation module.
[0285] (III) Attack Strategy Generation Module
[0286] 1. A policy generation model based on reinforcement learning is adopted, in which the policy network adopts a deep neural network structure, and the input is the state representation. The output is a probability distribution of adversarial test actions (i.e., adversarial test text). Specifically, this is achieved through sampling or... Choose adversarial test actions The training objective of the policy network is to maximize the cumulative reward objective function. .
[0287] here, ;
[0288] in, As an attack strategy, For the attack trajectory, In time step The reward This is the discount factor.
[0289] 2. Reward Function Design
[0290] The excitation function comprehensively considers the matching degree between the attack content and the test target, the degree of sensitive information leakage, and the model response length. Specifically:
[0291] ;
[0292] in, In time step Attack samples, For the test target, , , These are the weight parameters.
[0293] here, This indicates the degree of match between the attack content and the target.
[0294] ;
[0295] here, This indicates the degree to which sensitive information has been leaked.
[0296] ;
[0297] in, Indicates attack text Sensitive words in For a predefined set of sensitive words, This is an indicator function.
[0298] here, This indicates the model response length.
[0299] ;
[0300] in, The length of the model response. The length of the target being attacked.
[0301] 3. Attack Feasibility Assessment
[0302] The formula for calculating the predicted attack success rate is as follows: .
[0303] in, for function, and For hyperparameters, This is the connection vector.
[0304] Furthermore, an attack strategy can be selected based on the feasibility assessment, specifically referring to the following formula:
[0305]
[0306] Among them, the feasibility threshold is When feasibility is high, the attack strategy with the highest Q value is selected. When feasibility is low, an attack strategy is randomly selected to increase the exploratory nature of the process.
[0307] Furthermore, the adversarial test actions generated by the policy network are converted into specific adversarial test texts, which are then preliminarily optimized to generate the final adversarial test texts. .
[0308] (iv) Attack Execution Module
[0309] The generated adversarial test text is applied to the target large language model to obtain the model's response. Combined with historical interactions, a dialogue context containing historical interactions is constructed. .
[0310] Furthermore, by reducing the probability of the attack text being detected through adversarial perturbations, adversarial test text with concealment is obtained. .
[0311] Among them, For the disturbance intensity, To detect the loss function of the model, It can be calculated through backpropagation .
[0312] The processed adversarial test text Send to the target large language model and obtain the model response. , ,in, It is a large language model.
[0313] Regarding rate limiting, the batch request rate will be controlled within a preset percentage (e.g., 80%) of the target large language model's rate threshold. For models with unknown rate limits, the rate limit will be determined through a small number of trial requests before batch execution. Regarding the single-request term limit, the site will control the total number of terms in a single request ("adversarial test text + context") within a preset percentage (e.g., 90%) of the model's single-request term limit, reserving space for the model to generate responses.
[0314] Regarding the number of concurrent connections, the number of concurrent connections is fixed by using a connection pool to ensure that the number of concurrent connections does not exceed a preset percentage (such as 80%) of the model's allowed upper limit.
[0315] Regarding cost control, for paid interfaces, set an upper limit for the cost of a single request based on the batch testing budget, and then calculate the maximum number of keywords per request to avoid interruption due to exceeding the budget.
[0316] Regarding response timeout, the response timeout threshold is set to a preset duration (e.g., 5 seconds). If a complete response is not received within this duration after the request is sent, it is considered a timeout. Timeout requests are retried (e.g., 2 retries, each with a preset interval). If the timeout still occurs after the retries, the request is marked as pending and added to the supplementary testing queue. When multiple requests time out consecutively, the request sending rate is temporarily reduced. Once the timeout is alleviated, the low rate is maintained to continue testing.
[0317] (v) Feedback Optimization Module
[0318] Attack effectiveness assessment
[0319] The formula for calculating the success rate of a test is: .
[0320] in, This indicates the degree of matching between the model's response and the attack target. This indicates the degree of sensitive information leaked in the model response.
[0321] Furthermore, the attack strategy is updated using the Q-learning algorithm, with the specific update formula as follows:
[0322] ;
[0323] in, For state, For action, For the next state, This is the learning rate.
[0324] Furthermore, adopt Strategy for exploration and utilization balance:
[0325]
[0326] in, To improve the exploration rate, a balance between exploration and utilization is maintained to ensure the system can discover new and effective attack paths.
[0327] Save experience data for batch updates. .
[0328] It should be understood that, although Figure 2-9 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2-9 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.
[0329] In one embodiment, such as Figure 10 As shown, a testing device 100 for a large language model is provided, including: a historical dialogue analysis module 101, a test strategy generation module 102, and a test execution module 103, wherein:
[0330] The historical dialogue analysis module 101 is used to obtain the target historical dialogue in the conversation page and extract the historical dialogue features of the target historical dialogue; wherein, the conversation page is used to have a dialogue with the target large language model.
[0331] The test strategy generation module 102 is used to generate adversarial test text for testing the target historical dialogue based on historical dialogue features.
[0332] The test execution module 103 is used to input adversarial test text in the session page and obtain the test results of the target large language model's response to the adversarial test text.
[0333] In one embodiment, the testing apparatus 100 for the large language model further includes a historical dialogue accumulation module 104, which comprises a first update unit 1041, a second update unit 1042, and an accumulation unit 1043, wherein:
[0334] The first update unit 1041 is used to update the test results to the target historical dialogue in the conversation page, so as to obtain the updated target historical dialogue.
[0335] The second update unit 1042 is used to extract new historical dialogue features based on the updated target historical dialogue, generate new adversarial test text, and input the new adversarial test text into the conversation page to obtain new test results.
[0336] The first accumulation unit 1043 is used to repeat the above update and generation process to perform multi-turn dialogue tests on the target large language model under the conversation page to obtain multi-turn historical dialogues.
[0337] In one embodiment, the historical dialogue analysis module 101 includes a second accumulation unit 1011, wherein:
[0338] The second accumulation unit 1011 is used to acquire multi-turn historical dialogues in multiple dialogue windows to obtain the target historical dialogue.
[0339] In one embodiment, the historical dialogue analysis module 101 further includes an encoding unit 1012, a dialogue state tracking unit 1013, an extraction unit 1014, and a feature integration unit 1015, wherein:
[0340] The encoding unit 1012 is used to perform semantic encoding on the target historical dialogue to obtain the semantic representation of the target historical dialogue;
[0341] The dialogue state tracking unit 1013 is used to track the dialogue state of the target historical dialogue based on semantic representation to obtain the dialogue state; wherein, the dialogue state is used to represent the contextual information of the target historical dialogue.
[0342] Extraction unit 1014 is used to extract key information based on semantic representation; wherein, the key information includes at least one of user intent, dialogue topic and historical adversarial test pattern;
[0343] The feature integration unit 1015 is used to integrate semantic representation, dialogue state and key information to obtain historical dialogue features.
[0344] In one embodiment, the test policy generation module 102 includes a first conversion unit 1021, a policy network processing unit 1022, and a second conversion unit 1023, wherein:
[0345] The first conversion unit 1021 is used to convert historical dialogue features into state representations;
[0346] The policy network processing unit 1022 is used to input the state representation into the policy network based on reinforcement learning, and output the adversarial test action corresponding to the state representation based on the policy network.
[0347] The second conversion unit 1023 is used to convert adversarial test actions into text to obtain adversarial test text.
[0348] In one embodiment, the policy network processing unit 1022 includes an action generation subunit 1022a, an evaluation subunit 1022b, a first evaluation result subunit 1022c, and a second evaluation result subunit 1022d, wherein:
[0349] The action generation subunit 1022a is used to generate multiple candidate adversarial test actions based on the policy network and the state representation.
[0350] Evaluation subunit 1022b is used to evaluate the feasibility of multiple candidate adversarial test actions and obtain the predicted success rate value corresponding to the candidate adversarial test actions;
[0351] The first evaluation result subunit 1022c is used to select the candidate adversarial test action with the highest value when the predicted success rate is not lower than the feasibility threshold, and obtain the adversarial test action.
[0352] The second evaluation result subunit 1022d is used to randomly select from multiple candidate adversarial test actions when the predicted success rate is lower than the feasibility threshold, thus obtaining the adversarial test action.
[0353] In one embodiment, the test policy generation module 102 further includes a policy network generation unit 1024, which includes a training data acquisition subunit 1024a, a reward calculation subunit 1024b, and an optimization subunit 1024c, wherein:
[0354] Acquire subunit 1024a, which is used to acquire training data consisting of multiple test trajectories; wherein, the test trajectory includes the state representation of each time step, the adversarial test action and the corresponding reward;
[0355] The reward calculation subunit 1024b is used to calculate the reward at each time step based on the reward function; wherein, the reward function is determined based on at least one of the following: the matching degree between the test content and the test objective, the degree of leakage of sensitive information, and the model response length.
[0356] The optimization subunit 1024c is used to update the parameters of the policy network with the goal of maximizing the cumulative reward of the test trajectory, thus obtaining the updated policy network.
[0357] In one embodiment, the test execution module 103 includes a splicing unit 1031, a disturbance application unit 1032, and a response unit 1033, wherein:
[0358] The splicing unit 1031 is used to splice the adversarial test text with the target historical dialogue to construct the first dialogue context;
[0359] The perturbation application unit 1032 is used to apply adversarial perturbation to the adversarial test text in the first dialogue context to obtain the second dialogue context;
[0360] The response unit 1033 is used to input the second dialogue context into the conversation page and obtain the test results of the target large language model's response to the adversarial test text.
[0361] In one embodiment, the testing apparatus 100 for a large language model further includes a batch processing module 105, which includes a filtering unit 1051, a timeout determination unit 1052, and a sending unit 1053, wherein:
[0362] The filtering unit 1051 is used to filter multiple test requests according to the request constraints of the target large language model to obtain the target test request; wherein the request constraints include at least one of rate limit, single word limit and concurrent connection number;
[0363] The timeout determination unit 1052 is used to determine that the target test request has timed out when the response time of the target test request exceeds a preset timeout threshold.
[0364] The sending unit 1053 is used to reduce the sending rate of target test requests when multiple consecutive target test requests time out.
[0365] In one embodiment, the testing apparatus 100 for the large language model further includes an iterative optimization module 106, which comprises a computation unit 1061, an evaluation unit 1062, an empirical data generation unit 1063, and a policy network iteration unit 1064, wherein:
[0366] The calculation unit 1061 is used to calculate the success rate of the test based on the test results;
[0367] Evaluation unit 1062 is used to evaluate the test effectiveness of adversarial test text based on test success rate;
[0368] The experience data generation unit 1063 is used to generate and store experience data representing the test process based on the test results.
[0369] The policy network iteration unit 1064 is used to optimize the policy network based on empirical data, which determines the adversarial test actions corresponding to the adversarial test text according to the features of historical dialogues.
[0370] In one embodiment, the policy network iteration unit 1064 includes a reward determination subunit 1064a, a value estimation subunit 1064b, and an action update subunit 1064c, wherein:
[0371] The reward determination subunit 1064a is used to determine the reward corresponding to the adversarial test action based on the test results.
[0372] Value estimation subunit 1064b is used to update the value estimate of selecting adversarial test actions under the state representation based on the reward;
[0373] The action update subunit 1064c is used to select the adversarial test action with the highest value based on the value estimate with a first probability, and to randomly select the adversarial test action with a second probability.
[0374] Specific limitations regarding the testing apparatus for large language models can be found in the limitations of the testing methods for large language models described above, and will not be repeated here. Each module in the aforementioned testing apparatus for large language models can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0375] In one embodiment, a communication device is provided, see [link to previous document]. Figure 11 , Figure 11 This is a schematic diagram of the structure of the terminal device provided in an embodiment of the present invention. Figure 11 The terminal device 700 shown includes at least one processor 701, a memory 702, at least one network interface 704, and a user interface 703. The various components in the terminal device 700 are coupled together via a bus 705. It is understood that the bus 705 is used to implement communication between these components. In addition to a data bus, the bus 705 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 1 In this embodiment, all buses are designated as bus 705. Additionally, this embodiment also includes a transceiver 706, which may consist of multiple components, including a transmitter and a receiver, providing a unit for communicating with various other devices over a transmission medium.
[0376] The user interface 703 may include a display, keyboard, or clicking device (e.g., mouse, trackball, touchpad, or touchscreen).
[0377] It is understood that the memory 702 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 702 of the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.
[0378] In some implementations, memory 702 stores elements such as executable modules or data structures, or subsets thereof, or extended sets thereof: operation 7021 and application program 7022.
[0379] Operation 7021 includes various programs, such as a framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. Application program 7022 includes various applications, such as a media player and browser, used to implement various application business functions. The program implementing the method of the embodiments of the present invention can be included in application program 7022.
[0380] In this embodiment of the invention, by calling the program or instructions stored in memory 702, specifically the program or instructions stored in application program 7022, the processor is used to obtain the target historical dialogue in the conversation page and extract the historical dialogue features of the target historical dialogue; wherein the conversation page is used to have a dialogue with the target large language model; based on the historical dialogue features, adversarial test text is generated to test the target historical dialogue; the adversarial test text is input in the conversation page, and the test results of the target large language model's response to the adversarial test text are obtained.
[0381] The methods disclosed in the above embodiments of the present invention, in part or in all of them, can also be applied to processor 701, implemented by processor 701, or implemented by processor 701 in conjunction with other components (e.g., transceivers). Processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above methods can be completed by the integrated logic circuit of the hardware in processor 701 or by instructions in the form of software. The processor 701 described above may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 702, and processor 701 reads the information from memory 702 and, in conjunction with its hardware, completes the steps of the above method.
[0382] It is understood that the embodiments described in these embodiments of the present invention can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions of this application, or combinations thereof.
[0383] For software implementation, the techniques of the embodiments of the present invention can be implemented through modules (e.g., procedures, functions, etc.) that perform the functions of the embodiments of the present invention. The software code can be stored in memory and executed by processor 701. The memory can be implemented in processor 701 or external to processor 701.
[0384] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of a testing method for a large language model.
[0385] This application also provides a computer program product containing instructions that, when run on a computer, cause the steps performed by the computer large language model testing method.
[0386] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.
[0387] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0388] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
Claims
1. A testing method for a large language model, characterized in that, The method includes: Obtain the target historical dialogue from the conversation page and extract the historical dialogue features of the target historical dialogue; wherein, the conversation page is used to engage in dialogue with the target large language model; Based on the historical dialogue features, generate adversarial test text to test the target historical dialogue; Enter the adversarial test text into the session page and obtain the test results of the target large language model's response to the adversarial test text.
2. The method according to claim 1, characterized in that, The target historical dialogue includes multiple rounds of historical dialogue on the session page, and the method further includes: The test results are then updated to the target historical dialogue in the conversation page to obtain the updated target historical dialogue. Based on the updated target historical dialogue, new historical dialogue features are extracted, and new adversarial test text is generated. The new adversarial test text is then input into the conversation page to obtain new test results. Repeat the above update and generation process to perform multi-turn dialogue tests on the target large language model under the conversation page to obtain the multi-turn historical dialogue.
3. The method according to claim 1, characterized in that, The session page includes multiple dialog windows, each containing multiple rounds of historical dialogue. The step of obtaining the target historical dialogue from the session page includes: The target historical dialogue is obtained by acquiring the multi-turn historical dialogue from multiple dialogue windows.
4. The method according to claim 1, characterized in that, The step of extracting the historical dialogue features of the target historical dialogue includes: Semantic encoding is performed on the target historical dialogue to obtain the semantic representation of the target historical dialogue; Based on the semantic representation, the dialogue state of the target historical dialogue is tracked to obtain the dialogue state; wherein, the dialogue state is used to represent the contextual information of the target historical dialogue; Based on the semantic representation, key information is extracted; wherein, the key information includes at least one of user intent, dialogue topic, and historical adversarial test patterns; The semantic representation, the dialogue state, and the key information are integrated to obtain the historical dialogue features.
5. The method according to claim 1, characterized in that, The step of generating adversarial test text for testing the target historical dialogue based on the historical dialogue features includes: Convert the historical dialogue features into state representations; The state representation is input into a reinforcement learning-based policy network, and the policy network outputs an adversarial test action corresponding to the state representation. The adversarial test actions are converted into text to obtain the adversarial test text.
6. The method according to claim 5, characterized in that, The step of inputting the state representation into a reinforcement learning-based policy network and outputting an adversarial test action corresponding to the state representation based on the policy network includes: Based on the policy network, multiple candidate adversarial test actions are generated according to the state representation; A feasibility assessment is performed on multiple candidate adversarial test actions to obtain the predicted success rate value corresponding to the candidate adversarial test actions; When the predicted success rate is not lower than the feasibility threshold, the candidate adversarial test action with the highest value is selected to obtain the adversarial test action; When the predicted success rate is lower than the feasibility threshold, the adversarial test action is randomly selected from a plurality of candidate adversarial test actions.
7. The method according to claim 5, characterized in that, The steps for generating the policy network include: Acquire training data consisting of multiple test trajectories; wherein the test trajectories include state representations at each time step, adversarial test actions, and corresponding rewards; The reward for each time step is calculated based on the reward function; wherein the reward function is determined based on at least one of the following: the matching degree between the test content and the test objective, the degree of leakage of sensitive information, and the model response length. With the goal of maximizing the cumulative reward of the test trajectory, the parameters of the policy network are updated to obtain the updated policy network.
8. The method according to claim 1, characterized in that, The step of inputting the adversarial test text into the session page and obtaining the test result of the target large language model's response to the adversarial test text includes: The adversarial test text is concatenated with the target historical dialogue to construct a first dialogue context; An adversarial perturbation is applied to the adversarial test text in the first dialogue context to obtain a second dialogue context; The second dialogue context is input into the conversation page, and the test result of the target large language model responding to the adversarial test text is obtained.
9. The method according to claim 1, characterized in that, If there are multiple test requests, before entering the adversarial test text on the session page, the method further includes: Based on the request constraints of the target large language model, multiple test requests are filtered to obtain the target test request; wherein, the request constraints include at least one of rate limit, single-time word limit, and concurrent connection number; When the response time of the target test request exceeds a preset timeout threshold, the target test request is determined to have timed out. If multiple consecutive target test requests time out, reduce the sending rate of the target test requests.
10. The method according to claim 1, characterized in that, After obtaining the test results of the target large language model's response to the adversarial test text, the method further includes: Calculate the test success rate based on the test results; The effectiveness of the adversarial test text is evaluated based on the test success rate. Based on the test results, generate and store empirical data to represent the test process. Based on the empirical data, optimize the policy network that determines the adversarial test actions corresponding to the adversarial test text according to the historical dialogue features.
11. The method according to claim 10, characterized in that, The empirical data includes at least the state representation corresponding to the historical dialogue features, the adversarial test actions, and the test results. The step of optimizing the policy network based on the empirical data to determine the adversarial test actions corresponding to the adversarial test text according to the historical dialogue features includes: Based on the test results, determine the reward corresponding to the adversarial test action; Based on the reward, update the value estimate of selecting the adversarial test action under the state representation; The adversarial test action with the highest value is selected based on the value estimate with a first probability, and the adversarial test action is randomly selected with a second probability.
12. A testing device for a large language model, characterized in that, The device includes: The historical dialogue analysis module is used to obtain the target historical dialogue in the conversation page and extract the historical dialogue features of the target historical dialogue; wherein, the conversation page is used to have a dialogue with the target large language model; The test strategy generation module is used to generate adversarial test text for testing the target historical dialogue based on the historical dialogue features. The test execution module is used to input the adversarial test text into the session page and obtain the test results of the target large language model's response to the adversarial test text.
13. A communication device, characterized in that, The method includes a memory and a processor, the memory storing a computer program, characterized in that the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 11.
14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.
15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.
