Multi-level visual analysis system for big language model red team drilling

Through a multi-level visual analysis system, the shortcomings of adversarial mode and volatility analysis in the red team drill of the large language model are solved, and detailed visual design and interactive interface are provided, which improves the security of the model and the efficiency of the red team. Significantly improves the robustness and interpretability of the model.

CN119088951APending Publication Date: 2024-12-06ZHEJIANG UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410973628.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing large language model Red Team drill technology has the adversarial pattern presentation and fluctuation analysis that is not comprehensive enough, and it is difficult to effectively support the behavior analysis and robustness evaluation of the model during the training process. Moreover, traditional visualization methods cannot effectively reflect the uncertainty of the language model. Sex and volatility.

Method used

A multi-level visual analysis system for large language model red team drills was designed, including control panel, embedded analysis, indicator monitoring, adversarial flow, volatility analysis and instance list modules, which demonstrate adversariality through multi-level views and interactive interfaces. , and volatility, provide detailed performance metrics and volatility analysis, enhancing the interpretability and security of the model.

Benefits of technology

The system provides a comprehensive view of adversarial dynamics, assessing the possibility of token transitions in model answers, helping to determine the risk of model susceptibility to fraud and generating inappropriate replies, significantly improving the security of large language models and the efficiency of red teams.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088951A_ABST
    Figure CN119088951A_ABST
Patent Text Reader

Abstract

The invention discloses a multi-level visual analysis system for big language model red team drilling, and the system comprises a control panel module which comprises model basic information, model management and training control buttons; the embedding analysis module comprises an embedding distribution scatter diagram of the text data set generated in the iteration process; the index monitoring module is used for displaying the overall performance index of the model through a radar map and displaying the change trend of the index in the iteration process through a biaxial broken line graph; the adversarial flow module displays the performance change of the target model for processing different types of adversarial prompts through a three-level view; the volatility analysis module comprises a list of volatility analysis of a prompt level and a token level on the verification set; the instance list module displays a detailed information list of the instance set; and the rear-end calculation module is used for carrying out iterative training and verification on the red team model and the target model, calculating related indexes, and carrying out data interaction with all other modules to realize real-time updating of data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of visualization of large language model security, and in particular relates to a multi-level visual analysis system for large language model red team exercises. Background Art

[0002] Large language models, such as ChatGPT and LLaMA, have complex structures and a large number of parameters. Their powerful computing power and rich knowledge reserves make them powerful in natural language processing and have become general artificial intelligence solutions in various application fields. But at the same time, ethical and security issues of large language models have also emerged. Some incorrect and malicious use of large language models may cause serious harm. Developing safe and reliable large language models has become an important social responsibility of model providers.

[0003] To address these issues, people use various techniques to align large language models with human values, such as using reinforcement learning based on human feedback to prevent the model from producing inappropriate outputs. These techniques help improve the security of large language models, but there are still malicious people who try to bypass security firewalls to attack large language models. Red teaming is a common security technology in the field of artificial intelligence to deal with this problem. It constructs adversarial examples to test models to identify and fix vulnerabilities in artificial intelligence models. Traditional manual red teams often require a lot of labor costs. In order to improve efficiency and scalability, many works have begun to study the automatic generation of adversarial datasets. Using large language models as generators of adversarial prompts (called "red models") is an effective method. However, existing work still has some unresolved difficulties, such as often tending to generate a single type of prompts in continuous training, or requiring a lot of time to converge. By introducing visual analysis, human wisdom can be incorporated to enhance the performance and interpretability of automatic red teams. However, the introduction of visual analysis still has the following two major challenges:

[0004] Presentation of adversarial patterns. The adversarial training process involves a subtle relationship between how the red team model generates adversarial cues and how the target large language model responds to them. While existing research has provided visual analysis techniques for reinforcement learning, they mainly focus on descriptive metrics such as training loss, diversity, or embeddings. Red teams need to focus on the behavior of the model throughout the training process. In addition, red teams often use dozens of training iterations, which raises scalability issues on how to show overall trends while supporting specific cues that identify model failures.

[0005] Support for fluctuation analysis. Simply using well-defined metrics to evaluate model performance, such as attack success rate, has certain limitations. This cannot reflect the robustness of the model. For example, for the same attack prompt, the model may respond appropriately to some checkpoints but not to other checkpoints, which means that the attacker may make the attack successful through certain interventions. Existing visual analysis methods for natural language processing usually understand model outputs by visualizing distributions. However, considering that large language models are sequence models, their outputs are heavily dependent on contextual information, and fluctuations in the first few words may greatly affect subsequent outputs. Existing related methods do not take into account the fluctuations of language models, especially in security scenarios. How to visualize uncertainty and support analysis of the probability of fluctuations is a challenge. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention proposes a multi-level visual analysis system for red team exercises of large language models. The system supports multi-level presentation of adversarial patterns and fluctuation analysis of models in the red team process of large language models.

[0007] The purpose of the present invention is achieved through the following technical solutions:

[0008] The embodiment provides a multi-level visual analysis system for large language model red team exercises, including:

[0009] The control panel module includes model import and export buttons, basic model information, and configurable model training related information;

[0010] An embedding analysis module, which includes a scatter plot of the high-dimensional embedding distribution of the text dataset produced by the target large language model in a single iteration in the semantic space;

[0011] The indicator monitoring module displays the overall performance indicators of a single iteration through a radar chart, and displays the trend tracking of the indicators changing over time during the iteration process through a dual-axis line chart;

[0012] The Adversarial Flow module shows the performance changes of the target large language model on the adversarial prompt dataset through a three-level view;

[0013] The volatility analysis module includes a list of volatility analyses at both the prompt level and the token level on the validation set, as well as a detail card for a prompt that appears when you click on it;

[0014] The instance list module displays a detailed list of the selected instance set and interacts with other views to provide users with supplementary details. The detailed information includes the specific adversarial prompt text, the risk type label of the prompt, and the response type label of the target large language model.

[0015] The back-end computing module will train the red team model, perform iterative red team training and verification on the target large language model, calculate relevant indicators of the red team results, and interact with the control panel module, embedded analysis module, indicator monitoring module, adversarial flow module, volatility analysis module, and instance list module to achieve real-time data updates.

[0016] Preferably, in the control panel module, the configurable model training related information includes an input box for setting the temperature parameters and the number of iterations of model training, a pie chart for visualizing the seed ratio, and an interface for model management and training process customization provided to the user in an interactive manner, wherein the seed ratio pie chart contains two parts, namely, the seed pool ratio and the seed sampling ratio, and the required seed sampling ratio is fed back by adjusting the size of each part in the pie chart.

[0017] Preferably, in the embedding analysis module, the text dataset includes the potential feature distribution of the dialogue text generated by the target large language model under the challenge of adversarial prompts of the training set, the validation set and the entire set. For the scatter plot, the user is supported to select specific data points from the scatter plot and add them to the seed pool.

[0018] Preferably, in the indicator monitoring module, the indicators displayed in the dual-axis line chart are custom selected by clicking on the radar chart label for interaction.

[0019] Preferably, in the indicator monitoring module, the performance indicators include the toxicity score of the target large language model on the training set, the attack success rate on the training set, the toxicity score of the large language model on the verification set, the attack success rate on the verification set, the diversity of adversarial prompt texts generated by the red team model, and the answer accuracy of the target large language model in general reasoning tasks.

[0020] Preferably, in the adversarial flow module, the adversarial prompt data set includes two parts: a training set and a validation set; the three-level view includes a first-level view, which is a stacked area chart showing the overall change of reaction types on the training set and a Sankey diagram showing the overall change of reaction types on the validation set; a second-level view, which is a Sankey diagram showing the distribution relationship between the risk type prompted in each round of training and the reaction type of the model; and a third-level view, which is a component change card of a specific type of data set in the training set and a fluctuation path of a specific data in the validation set in terms of reaction type;

[0021] The interactive method of the three-level view is: use the mouse wheel to zoom in to the second-level view on the first-level view, and click on the second-level view to display the third-level view.

[0022] Preferably, in the third-level view, the component change cards of a specific type of data set are divided into two types, namely, a stacked bar chart displayed after clicking the Sankey diagram node of the second-level view, reflecting the internal proportion change; and a line chart displayed after clicking the flow of the Sankey diagram of the second-level view, reflecting the size change of the data set;

[0023] The volatility path shown in the Sankey diagram of the validation set is a line element that runs through the entire process. Its position in each iteration represents the type of response made by the target large language model. If the target large language model makes a safe response, the radius of a circular node will be used to represent the token-level volatility in this round of iteration.

[0024] Preferably, in the volatility analysis module, the list includes the prompt-level volatility value, token-level volatility average, and token-level volatility change trend of each prompt; the detail information card includes specific prompts, specific answers, model answer tokens and their generation probability, and risk tokens and their generation probability;

[0025] Among them, the prompt-level volatility calculates the frequency of the target large language model's transition between different response types on an adversarial prompt; the token-level volatility calculates the distance between the safe response token and the hazard response token generated by the target large language model, and evaluates the risk of the safe response turning into a hazard response;

[0026] In the details card, you can modify the model's answer in the specific answer text box and feed it back to the target large language model for learning.

[0027] Preferably, the instance list module interacts with the volatility analysis module, the embedded analysis module, and the adversarial flow module. After clicking a specific card in the instance list, the selected item will be highlighted in the list of the volatility analysis module, the specific location of the prompt will be displayed in the embedded analysis module, and the specific volatility path of the prompt will be displayed in the adversarial flow module.

[0028] Preferably, in the back-end computing module, the model training part uses a reinforcement learning architecture, and the trained red team model and the target large language model will obtain feedback from the evaluation made by the reward model and perform iterative adjustment training.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] The multi-level visual analysis system of the present invention proposes a visualization design of multi-level adversarial flows, which provides a comprehensive view of adversarial dynamics, from an overview of the entire training process to granular details of a single epoch. In addition, a volatility metric is designed to evaluate the likelihood of token transitions in model responses, which helps to determine how large language models are susceptible to being deceived and producing inappropriate responses, and clearly describes the uncertainty in the process. The present invention provides a new visual analysis method for the red team, which effectively enhances the security of large language models. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0032] Figure 1 It is a schematic diagram of the structure of a multi-level visual analysis system provided in an embodiment. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0034] like Figure 1 As shown, the embodiment provides a multi-level visual analysis system for large language model red team exercises, including a control panel module, an embedded analysis module, an indicator monitoring module, an adversarial flow module, a volatility analysis module, an instance list module and a backend calculation module. In the interface of the visual analysis system, A is a control panel module: A1 is a basic information and model control button; A2 is a seed data set ratio pie chart, B is an embedded analysis module, C is an indicator monitoring module: C1 is a radar chart of the overall performance indicators under the current iteration round; C2 is a double-axis line chart of the change trend of the two selected indicators, D is an adversarial flow module: D1 is a training set flow; D2 is a type of data set detail card displayed after clicking on a node or flow of the verification set flow; D3 is a verification set flow; D4 is a corresponding volatility path displayed after clicking on a specific prompt, E is an instance list module, and F is a volatility analysis module: F1 is an information card corresponding to each prompt in the list; F2 is a detailed information card under the selected iteration round; F3 is an answer token, a risk token and its probability display. The following is a detailed description of each module.

[0035] The control panel module A includes basic information of the imported model, buttons for importing and exporting the model, and input boxes for setting the temperature parameters and the number of iterations for model training ( Figure 1 A1 in the middle), pie chart to visualize seed ratio ( Figure 1 In A2), an interface for model management and training process customization is provided to users in an interactive manner.

[0036] For example, before starting formal training, you can use the model import button in the control panel module A to import the local model file as the target large language model, then set the number of iterations and temperature parameters in the iteration number and temperature parameter setting input box, and click the start button to start red team training. After the training is completed, you can adjust the sampling ratio in the seed sampling ratio pie chart based on the analysis results. If you want to end the training, you can click the export button to export the target large language model that has been trained by the red team security.

[0037] Embedding analysis module B includes a scatter plot of the high-dimensional embedding distribution of the text dataset in the semantic space generated in a specific iteration ( Figure 1 B); the text dataset includes the potential feature distribution of the dialogue text generated by the target large language model under the challenge of adversarial prompts in the training set, validation set and full set. The dataset can be switched by clicking the button.

[0038] The selected small data subset in the system will be highlighted in the scatter plot. In addition, the embedded analysis view provides a selection function, where users can select important data points in the scatter plot corresponding to the analysis results and click a button to add them to the seed pool set, so that the target large language model can increase the training of the type of data that can be used.

[0039] The indicator monitoring module C displays the overall performance indicators of this iteration through a radar chart ( Figure 1 C1), and a dual-axis line chart showing the trend of indicators changing over time during the iteration process ( Figure 1 C2); the performance indicators include the toxicity score of the target large language model on the training set, the attack success rate on the training set, the toxicity score of the target large language model on the validation set, the attack success rate on the validation set, the diversity of the adversarial prompt texts generated by the red team model, and the answer accuracy of the target large language model in general reasoning tasks.

[0040] The indicators displayed in the dual-axis line chart are customized by clicking on the radar chart label to interact. The dual-axis line chart facilitates users to compare and analyze different performance indicators and find the required performance balance node.

[0041] The adversarial flow module D shows the performance changes of large language models on adversarial prompt datasets through a three-level view; the adversarial datasets include training sets ( Figure 1 D1), validation set ( Figure 1 D3) in two parts; the three-level view includes: the first level view is a stacked area chart showing the overall change of reaction types on the training set and a Sankey diagram showing the overall change of reaction types on the validation set ( Figure 1 D3 in the middle), the second level view is a Sankey diagram showing the distribution relationship between the risk type prompted in each round of training and the response type of the model ( Figure 1 D1 in the middle), the third level view is the composition change card of a specific type of data set in the training set ( Figure 1 D2) and the fluctuation path of a specific data in the validation set on the reaction type ( Figure 1 in D4).

[0042] The three-level view is progressively advanced through mouse interaction. The interaction method is to zoom in to the second-level view by using the mouse wheel on the first-level view, and click the node or flow of the Sankey diagram on the second-level view to display the third-level view. In order to enhance scalability, a horizontal scroll bar is set in the adversarial flow module, and users can drag the scroll bar to locate the round of interest.

[0043] Among them, in the third-level view of the adversarial flow module, there are two types of cards for changes in the composition of specific types of data sets. One is a stacked bar chart displayed after clicking the second-level Sankey diagram node, which reflects the internal proportion changes of the selected type of data set; the other is a line chart displayed after clicking the flow of the second-level Sankey diagram, which reflects the changes in the size of the data set in which the model responds specifically to specific risk types.

[0044] In addition, the volatility path shown in the Sankey diagram of the validation set is a line element that runs through the entire process. Its position in each iteration represents the type of response made by the target large language model. If the target large language model makes a safe response, the radius of a circular node will be used to represent the token-level volatility in this round of iteration.

[0045] The volatility analysis module F includes the analysis of volatility at two levels, prompt level and token level, on the validation set ( Figure 1 F1), and click the details card of a prompt that appears when you interact ( Figure 1 F2); the list includes the prompt-level volatility value, token-level volatility average, and token-level volatility change trend of the prompt; the detailed information card includes specific prompts, specific answers, model answer tokens and their generation probability, risk tokens and their generation probability ( Figure 1 (F3).

[0046] Among them, the prompt-level volatility calculates the frequency of the target large language model's transition between different response types on an adversarial prompt; the token-level volatility calculation model generates the distance between the safe response token and the hazard response token, and evaluates the risk of the safe response turning into a hazard response.

[0047] In the details card, you can modify the model's answer in the text box of the specific answer to the answer that the user thinks is better. These answers will be fed back to the target large language model for learning.

[0048] The volatility analysis module F interacts with the embedded analysis module B and the adversarial flow module D. After clicking a specific card in the list, the selected item will be highlighted in the list of the volatility analysis module F, the specific location of the prompt will be displayed in the embedded analysis module B, and the specific volatility path of the prompt will be displayed in the adversarial flow module D.

[0049] The instance list module E displays the detailed information list of the selected instance set ( Figure 1 E), and interacts with other views to provide users with detailed supplementary display; the detailed information includes the specific adversarial prompt text, the risk type label of the prompt, and the response type label of the target large language model.

[0050] The instance list module E interacts with the volatility analysis module F, the embedded analysis module B, and the adversarial flow module D. After clicking a specific card in the instance list, the selected item will be highlighted in the list of the volatility analysis module F, the specific location of the prompt will be displayed in the embedded analysis module B, and the specific volatility path of the prompt will be displayed in the adversarial flow module D.

[0051] The back-end computing module will train the red team model, perform iterative red team training and verification on the target large language model, calculate relevant performance indicators and volatility indicators of the red team results, and interact with the control panel module A, embedded analysis module B, indicator monitoring module C, adversarial flow module D, volatility analysis module F, and instance list module E to achieve real-time data updates.

[0052] The model training part of the backend computing module uses a reinforcement learning architecture. The red team model generates adversarial prompts based on seed prompts and instructions, and the target model responds to the adversarial prompts generated by the red team. The reward model evaluates whether the target model responds appropriately to the adversarial prompts, and updates and fine-tunes the red team model and the target model based on the evaluation results as toxicity scores. In addition, the update and fine-tuning effects of the red team model and the target model are improved by automatically or manually adjusting the strategy or adding seed prompts and response pairs.

[0053] Specifically, the toxicity score is calculated as:

[0054] s=g(X ′ ,R)

[0055] Among them, g represents the reward model, s represents the toxicity score, s∈[-10,10] represents the response toxicity from highly toxic to non-toxic, and X ′ represents the adversarial cue word, and R represents the response generated by the target model.

[0056] Update and fine-tune the red team model and target model based on the assessment results, including:

[0057] According to the toxicity score, the adversarial prompts and response pairs are divided into a successful attack group and a failed attack group; the adversarial prompts and response pairs in the successful attack group are used to fine-tune the red team model to generate more effective attacks; the adversarial prompts and response pairs in the failed attack group are used to fine-tune the target model to enhance its defense capabilities; the adversarial prompts with high toxicity scores in the failed attack group are added to the seed database as seed prompts for future iterations; the adversarial prompts with the highest toxicity scores in the failed attack group are modified to refuse to answer, and then the target model is fine-tuned.

[0058] The above system comprehensively analyzes the security red team process of large language models, and adopts unique multi-level adversarial flow visualization and fluctuation path visualization technology to promote the efficiency of improving the security of the target large language model in the red team process.

[0059] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-level visual analysis system for large language model red team exercises, characterized by: include: The control panel module includes model import and export buttons, basic model information, and configurable model training related information; An embedding analysis module, which includes a scatter plot of the high-dimensional embedding distribution of the text dataset generated by the target large language model in a single iteration in the semantic space; The indicator monitoring module displays the overall performance indicators of a single iteration through a radar chart, and displays the trend tracking of the indicators changing over time during the iteration process through a dual-axis line chart; The Adversarial Flow module shows the performance changes of the target large language model on the adversarial prompt dataset through a three-level view; The volatility analysis module includes a list of volatility analyses at both the prompt level and the token level on the validation set, as well as a detail card for a prompt that appears when you click on it; The instance list module displays a detailed list of the selected instance set and interacts with other views to provide users with supplementary details. The detailed information includes the specific adversarial prompt text, the risk type label of the prompt, and the response type label of the target large language model. The back-end computing module will train the red team model, perform iterative red team training and verification on the target large language model, calculate relevant indicators of the red team results, and interact with the control panel module, embedded analysis module, indicator monitoring module, adversarial flow module, volatility analysis module, and instance list module to achieve real-time data updates.

2. The multi-level visual analysis system for large language model red team exercises according to claim 1, characterized in that: In the control panel module, the configurable model training related information includes the setting input box of the temperature parameters and the number of iterations of model training, the pie chart of the visualized seed ratio, and an interface for model management and training process customization provided to the user in an interactive manner. The seed ratio pie chart contains two parts, namely the seed pool ratio and the seed sampling ratio. The required seed sampling ratio is fed back by adjusting the size of each part in the pie chart.

3. The multi-level visual analysis system for large language model red team exercises according to claim 1, characterized in that: In the embedding analysis module, the text dataset includes the potential feature distribution of the dialogue text generated by the target large language model under the challenge of adversarial prompts of the training set, the validation set and the entire set. For the scatter plot, the user is supported to select specific data points from the scatter plot and add them to the seed pool.

4. The multi-level visual analysis system for large language model red team exercises according to claim 1, characterized in that: In the indicator monitoring module, the indicators displayed in the dual-axis line chart are customized by clicking on the radar chart label to interact.

5. The multi-level visual analysis system for large language model red team exercises according to claim 1 or 4, characterized in that: In the indicator monitoring module, the performance indicators include the toxicity score of the target large language model on the training set, the attack success rate on the training set, the toxicity score of the large language model on the verification set, the attack success rate on the verification set, the diversity of adversarial prompt texts generated by the red team model, and the answer accuracy of the target large language model in general reasoning tasks.

6. The multi-level visual analysis system for large language model red team exercises according to claim 1, characterized in that: In the adversarial flow module, the adversarial prompt data set includes two parts: a training set and a validation set; the three-level view includes a first-level view, which is a stacked area chart showing the overall change of reaction types on the training set and a Sankey diagram showing the overall change of reaction types on the validation set; a second-level view, which is a Sankey diagram showing the distribution relationship between the risk type prompted in each round of training and the reaction type of the model; and a third-level view, which is a component change card of a specific type of data set in the training set and a fluctuation path of a specific data in the validation set in terms of reaction type; The interactive method of the three-level view is: use the mouse wheel to zoom in to the second-level view on the first-level view, and click on the second-level view to display the third-level view.

7. The multi-level visual analysis system for large language model red team exercises according to claim 6, characterized in that: In the third-level view, there are two types of component change cards for a specific type of data set, namely, a stacked bar chart displayed after clicking a Sankey diagram node in the second-level view, reflecting the internal proportion change; and a line chart displayed after clicking a flow in the Sankey diagram in the second-level view, reflecting the size change of the data set; The volatility path shown in the Sankey diagram of the validation set is a line element that runs through the entire process. Its position in each iteration represents the type of response made by the target large language model. If the target large language model makes a safe response, the radius of a circular node will be used to represent the token-level volatility in this round of iteration.

8. The multi-level visual analysis system for large language model red team exercises according to claim 1, characterized in that: In the volatility analysis module, the list includes the prompt-level volatility value, token-level volatility average, and token-level volatility change trend of each prompt; the detail information card includes specific prompts, specific answers, model answer tokens and their generation probability, and risk tokens and their generation probability; Among them, the prompt-level volatility calculates the frequency of the target large language model's transition between different response types on an adversarial prompt; the token-level volatility calculates the distance between the safe response token and the hazard response token generated by the target large language model, and evaluates the risk of the safe response turning into a hazard response; In the details card, you can modify the model's answer in the specific answer text box and feed it back to the target large language model for learning.

9. The multi-level visual analysis system for large language model red team exercises according to claim 1, characterized in that: The instance list module interacts with the volatility analysis module, the embedded analysis module, and the adversarial flow module. After clicking a specific card in the instance list, the selected item will be highlighted in the list of the volatility analysis module, the specific location of the prompt will be displayed in the embedded analysis module, and the specific volatility path of the prompt will be displayed in the adversarial flow module.

10. The multi-level visual analysis system for large language model red team exercises according to claim 1, characterized in that: In the backend computing module, the model training part uses a reinforcement learning architecture. The trained red team model and the target large language model will obtain feedback from the evaluation made by the reward model and perform iterative adjustment training.

Citation Information

Cited By

  • Harmful cue word visual analysis system based on risk perception

    CN120995040A