Concept-guided large language model toxicity removal method based on sparse representation space

Abstract

The application belongs to the technical field of large language model security generation, and specifically discloses a large language model toxicity removal method based on concept guidance in a sparse representation space, which comprises the following steps: obtaining hidden representations of a large language model at a current decoding step, and encoding the hidden representations into sparse activation vectors through a sparse autoencoder; calculating concept activation scores according to a mapping set of toxicity concepts and sparse feature dimensions, and comparing the concept activation scores with a threshold to detect toxicity; when toxicity is detected, a dominant toxicity concept is selected, and a sparse detoxification direction of the dominant toxicity concept is used to perform directional inhibition intervention on the sparse activation vectors; and after the intervention, the sparse activation vectors are reconstructed through the sparse autoencoder and then fed back to the large language model for continuous generation. The application realizes fine-grained and accurate intervention on toxic semantics, effectively reduces toxic output, maintains the language fluency and general ability of the model, and has high reasoning efficiency.

Description

Technical Field

[0001] This invention belongs to the field of large language model safe generation technology, specifically involving a method for removing toxicity from large language models based on concept guidance within a sparse representation space. Background Technology

[0002] With the rapid development of artificial intelligence technology, Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding, generation, and reasoning. However, because they are typically pre-trained on large-scale, unfiltered online corpora, these models inevitably learn and internalize potentially harmful semantics and patterns. During the generation process, these harmful semantics may be triggered, leading to the model output containing offensive, biased, and other toxic content, thus affecting its reliability and security in real-world applications. Therefore, researching effective detoxification techniques is crucial for achieving the safe and controllable deployment of large language models.

[0003] Existing techniques for removing toxicity from large language models can be broadly categorized into two types: training-stage methods and inference-stage methods. Training-stage methods update model parameters by retraining or fine-tuning the large language model on a selected, non-toxic dataset, thereby reducing the probability of generating harmful content. Inference-stage methods, without altering the model parameters, suppress harmful content by intervening in the intermediate states or output distribution during the generation process.

[0004] However, existing technologies still have significant limitations. Training-phase methods typically rely on large-scale, high-quality labeled data, accompanied by high training costs and computational resource consumption. In practical applications, they are costly to deploy and lack flexibility, with limited adaptability in rapid iteration scenarios. Existing inference-phase representation editing methods usually rely on global or static intervention directions learned in the overall representation space, making it difficult to handle fine-grained differences between different toxic semantics. Furthermore, due to the highly superimposed nature of multiple semantics in the latent space of large language models, toxic and non-toxic semantics are often mixed and distributed within the same feature dimension. Coarse-grained interventions based on overall directions struggle to accurately distinguish and control different toxic concepts, resulting in limited detoxification effects. Summary of the Invention

[0005] The purpose of this invention is to address the problem that the lack of fine-grained distinction of toxic semantics affects the generality of the model. It proposes a toxicity removal method for large language models based on concept guidance within a sparse representation space.

[0006] The technical solution of this invention is as follows: This invention provides a method for removing toxicity from large language models based on concept guidance within a sparse representation space, comprising: Step 1: In the current decoding step, obtain the hidden representation of the current token by the large language model, and input the hidden representation into the pre-trained sparse autoencoder for encoding to obtain the corresponding sparse activation vector. Step 2: Calculate the activation score corresponding to each toxicity concept based on the sparse activation vector and the pre-constructed mapping set between toxicity concepts and sparse feature dimensions; Step 3: Compare the activation scores of each toxicity concept with a preset threshold. If the activation scores of all toxicity concepts do not exceed the preset threshold, it is determined that no toxic semantics were detected in the current decoding step, and the process proceeds to step 5. Otherwise, toxicity concepts with activation scores greater than the preset threshold are added to the set of activated concepts, and the toxicity concept with the highest activation score is selected as the dominant toxicity concept. Step 4: Based on the pre-determined sparse detoxification direction corresponding to each toxicity concept, determine the sparse detoxification direction of the dominant toxicity concept, and apply a directional inhibition operation to the sparse activation vector according to the sparse detoxification direction of the dominant toxicity concept to obtain the intervened sparse activation vector. Step 5: Input the sparse activation vector or the intervened sparse activation vector into the sparse autoencoder for reconstruction processing to obtain the adjusted hidden representation. Then, feed the adjusted hidden representation back to the large language model to continue subsequent token generation.

[0007] Preferably, the training process of the sparse autoencoder includes: S11: Input multiple paired labeled toxic and non-toxic responses into the large language model, extract the corresponding hidden representations, and form a training dataset; S12: Using the training dataset as the target, the sparse autoencoder is trained with the joint loss function consisting of reconstruction error and sparse regularization term, so that the sparse autoencoder learns sparse feature representations with discriminative ability for toxic semantics during the process of encoding hidden representations into sparse activation vectors; wherein, the joint loss function is expressed as:

[0008] in, For the original hidden representation, For the reconstruction of hidden representations by sparse autoencoders, For sparse activation vectors, This represents the sparsity weighting coefficient.

[0009] Preferably, the process of constructing the mapping set between the toxicity concept and the sparse feature dimension includes: S21: Use an auxiliary large language model to extract text fragments containing toxic semantics from toxic responses, and perform concept annotation on each text fragment according to a predefined set of toxic concepts to construct a concept-annotated toxic fragment dataset; S22: In the toxicity-induced scenario, the large language model performs autoregressive generation. When the generated token sequence matches any toxic fragment in the toxic fragment dataset, the concept label corresponding to the toxic fragment is recorded, and the hidden representation before the last token of the toxic fragment is extracted. S23: Input the extracted hidden representation into the sparse autoencoder to obtain the corresponding sparse activation vector, count the activation frequency of each sparse feature dimension under different concept labels, establish the mapping relationship between each toxicity concept and the corresponding sparse feature dimension, and form the mapping set.

[0010] Preferably, the predefined set of toxicity concepts includes: Obscene, Threat, Insult, and Identity_Hate.

[0011] As a preferred option, the activation score for each toxicity concept is calculated according to the following formula: ; In the formula, Toxicity concept Activation score, The concept of toxicity in mapping sets The corresponding set of feature dimensions, sparse activation vectors Dimensions activation value, This indicates the decoding step.

[0012] Preferably, the process for determining the sparse detoxification direction corresponding to each toxicity concept includes: S31: For each toxicity concept, collect multiple sample pairs formed by toxic and non-toxic samples, input the sample pairs into the large language model to extract the corresponding toxic hidden representation and non-toxic hidden representation, and input the toxic hidden representation and non-toxic hidden representation into the sparse autoencoder to encode the corresponding toxic sparse activation vector and non-toxic sparse activation vector. S32: Calculate the difference vector for each pair of samples based on the toxic sparse activation vector and the non-toxic sparse activation vector for each pair of samples. S33: Perform principal component analysis on all difference vector sets under each toxicity concept to determine the corresponding covariance matrix, extract the eigenvector corresponding to the largest eigenvalue of the covariance matrix as the first principal component, and determine the first principal component as the initial guiding direction of the corresponding toxicity concept. S34: Based on the feature dimension set corresponding to each toxicity concept in the mapping set, the initial guiding direction corresponding to each toxicity concept is sparsified to obtain the sparse detoxification direction of each toxicity concept.

[0013] Preferably, the sparsification process includes: for each toxicity concept, retaining the dimension values ​​of the feature dimension set corresponding to the toxicity concept in the initial guiding direction, and setting the remaining dimensions to zero.

[0014] Preferably, the sparse activation vector after intervention is calculated according to the following formula: ; In the formula, The sparse activation vector after intervention. For sparse activation vectors, This is the scaling factor. The sparse detoxification direction is the dominant concept of toxicity.

[0015] Preferably, the latent space dimension of the sparse autoencoder is larger than that of the latent space dimension of the large language model. The sparse autoencoder decomposes the dense hidden representation of the large language model into sparse representations through an overcomplete dictionary. The encoder of the sparse autoencoder uses ReLU or JumpReLU activation functions to achieve sparse activation.

[0016] The beneficial effects of this invention are: This invention presents a method for removing toxicity from large language models based on concept-guided representation space. By training a toxicity-oriented sparse autoencoder, the dense hidden representations of the large language model are decomposed into an interpretable sparse semantic feature space, and a mapping relationship is established between different toxic concepts and feature dimensions. During the inference phase, by real-time detection of the feature activation intensity corresponding to each toxic concept in the sparse activation vector, local selective intervention is performed only in the concept-related toxic subspace, achieving fine-grained toxicity suppression. Compared with existing technologies, this invention can effectively reduce toxic output while maintaining the model's original language generation capabilities and core general capabilities, without requiring additional training costs, and the inference speed is almost comparable to the original model. Attached Figure Description

[0017] Figure 1 The diagram shows the principle block diagram of a large language model toxicity removal method based on concept guidance in a sparse representation space. Figure 2 The diagram shows a flowchart of a large language model toxicity removal method based on concept guidance within a sparse representation space. Figure 3 The image shows a UMAP visualization of the four toxicity concepts in the characterization space. Figure 4 The figure shown is an experimental result graph showing the effect of the activated filter threshold on the toxicity score and generation fluency. Detailed Implementation

[0018] Exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the embodiments shown and described in the drawings are merely exemplary and are intended to illustrate the principles and spirit of the invention, and are not intended to limit the scope of the invention.

[0019] This invention provides a method for removing toxicity from large language models based on concept guidance within a sparse representation space. This method is applied to the inference stage of the large language model, which is an autoregressive language model based on the Transformer architecture, including but not limited to the GPT-2 series, Qwen series, and Gemma series. Please see [link to relevant documentation]. Figure 1 , Figure 1 The diagram shown illustrates the principle of a toxicity removal method for large language models based on concept guidance within a sparse representation space. Figure 1 As shown, the method of this invention encodes the intermediate hidden representations of a large language model using a pre-trained toxicity-oriented sparse autoencoder (SAE). In the sparse representation space, it combines the pre-constructed mapping relationship between toxicity concepts and sparse feature dimensions to construct specific intervention directions for different concepts. In the autoregressive generation process, toxicity detection and fine-grained intervention are performed, thereby suppressing toxic features while preserving non-toxic features as much as possible without causing damage to the core capabilities of the model. Finally, after reconstruction by the SAE decoder, it is fed back to the large language model for continued generation.

[0020] Please see Figure 2 , Figure 2 The diagram shows a flowchart of a toxicity removal method for large language models based on concept guidance within a sparse representation space. Figure 2 As shown, the toxicity removal method for large language models based on concept guidance within a sparse representation space in this embodiment includes the following steps: Step 1: In the current decoding step, obtain the hidden representation of the current token by the large language model, input the hidden representation into the pre-trained sparse autoencoder for encoding processing, and obtain the corresponding sparse activation vector.

[0021] Specifically, in the current decoding step Obtain the hidden representation of the current token from the large language model. ,in, This represents the latent space dimension of a large language model. It involves the hidden representations. The pre-trained sparse autoencoder is input for encoding to obtain the corresponding sparse activation vectors. ,in, Let be the latent space dimension of the sparse autoencoder. In this embodiment, the latent space dimension of the sparse autoencoder is much larger than the latent space dimension of the large language model, i.e. Sparse autoencoders decompose dense hidden representations of large language models into sparse representations using an overcomplete dictionary.

[0022] For example, taking GPT-2 small as an example, the latent space dimension of the original model is... The latent space dimension of SAE is 768. The value is 24576, corresponding to an expansion ratio of approximately 32. Because... SAE learns an overcomplete dictionary that can represent the input through a small number of activated feature combinations, while encouraging each latent dimension of SAE to learn more interpretable semantic patterns.

[0023] In this embodiment, the encoding and decoding processes of the sparse autoencoder are represented as follows: ; ; in, For encoding matrix, For the decoding matrix, For encoding bias, For decoding bias, For activation function, To reconstruct hidden representations.

[0024] Optionally, the encoder of the sparse autoencoder employs ReLU or JumpReLU activation functions to achieve sparse activation. Under the action of ReLU or JumpReLU activation functions, most dimensions of the encoder output are zero, thus achieving sparse activation. This makes the sparse activation vector... Only a few feature dimensions have non-zero activation values, which facilitates subsequent toxicity concept activation detection and fine-grained intervention operations. SAE can learn sparse and semantically structured latent feature representations while preserving the original information, providing a structured feature foundation for subsequent toxicity concept localization and intervention.

[0025] In this embodiment, the training process of the sparse autoencoder includes the following steps: S11: Input multiple paired labeled toxic and non-toxic responses into the large language model, extract the corresponding hidden representations, and form a training dataset.

[0026] Specifically, obtain paired labeled toxic and non-toxic responses. For a given hint... This corresponds to a non-toxic response. A toxic response The two responses are then input into a large language model to extract the hidden representations of its intermediate layers. and Together, these form the training dataset for SAE. Since paired responses are highly similar in contextual semantics but differ in toxicity attributes, their hidden representations can provide semantic change signals related to toxicity, enabling SAE to learn feature representations that distinguish between toxic and non-toxic substances during training.

[0027] S12: Using the joint loss function consisting of reconstruction error and sparse regularization term as the target, the sparse autoencoder is trained with the training dataset so that the sparse autoencoder learns sparse feature representations that can distinguish toxic semantics in the process of encoding hidden representations into sparse activation vectors.

[0028] In this embodiment, the joint loss function is expressed as:

[0029] in, For the original hidden representation, For the reconstruction of hidden representations by sparse autoencoders, For sparse activation vectors, This is the sparsity weighting coefficient, used to control the balance between reconstruction accuracy and sparsity. This is the reconstruction error term, used to measure the difference between the original hidden representation and the SAE reconstructed hidden representation; This is an L1 sparse regularization term used to constrain the sparsity of sparse activation vectors.

[0030] During training, by minimizing this joint loss function, SAE is able to learn sparse and semantically structured latent feature representations while preserving the original information.

[0031] Step 2: Calculate the activation score for each toxicity concept based on the sparse activation vector and the pre-built mapping set between toxicity concepts and sparse feature dimensions.

[0032] Understandably, common editing methods typically adjust spatial representations through static or coarse-grained interventions to guide the generation towards non-toxic regions in the latent space. These methods usually first identify a direction representing "toxic semantics" and then apply interventions to each token and representation. Because these methods do not explore the toxicity of different semantics in a fine-grained manner and often rely only on a single global vector for coarse-grained intervention, they may interfere with or destroy some features unrelated to toxic semantics, thus limiting the detoxification effect and potentially affecting the model's generalizability.

[0033] Based on this, embodiments of the present invention construct a mapping relationship between toxicity concepts and potential feature dimensions in a sparse representation space, and based on this, detect the toxicity information that may be contained in the representation during the autoregression process and perform fine-grained intervention, thereby achieving more accurate detoxification.

[0034] Specifically, the process of constructing the mapping set between the toxicity concept and sparse feature dimensions includes the following steps: S21: Use an auxiliary large language model to extract text fragments containing toxic semantics from toxic responses, and perform concept annotation on each text fragment according to a predefined set of toxic concepts to construct a concept-annotated toxic fragment dataset.

[0035] In this embodiment, based on the labeling system defined by the Jigsaw Toxic Comment Classification Challenge dataset, a predefined set of toxicity concepts is introduced, represented as follows: .

[0036] That is, the toxicity concept set includes four categories: Obscene, Threat, Insult, and Identity Hate.

[0037] Furthermore, the rationality of the concept classification was verified. On GPT2-small, 300 sentences were randomly selected from the toxicity data of each category, and their tail token representations were extracted. These representations were then projected onto a two-dimensional space using UMAP for visualization. (See [link to UMAP documentation]). Figure 3 , Figure 3 The diagram shows a UMAP visualization of the four toxicity concepts in the characterization space.

[0038] like Figure 3 As shown in the visualization, the results indicate that the latent representations of different toxicity concepts in the representation space exhibit a relatively separable clustering structure, demonstrating that the spatial distribution of different toxicity concepts in the latent space does indeed possess significantly different structural features, and that this classification set is reasonable. Theoretically, while maintaining the existing classification structure, the method of this invention also possesses good scalability for other toxicity concept categories with significant semantic distinguishability.

[0039] Specifically, the guided and assisted large language model classifies each text segment according to its semantic meaning. Assign to the corresponding concept tag By summarizing all labeled fragments, a concept-labeled toxic fragment dataset is constructed: ; In the formula, This represents the number of text segments.

[0040] This dataset serves as an intermediate semantic interface between natural language toxicity expressions and SAE latent features, providing centralized concept-level supervision signals for subsequent concept-feature association modeling.

[0041] S22: In the toxicity-induced scenario, the large language model performs autoregressive generation. When the generated token sequence matches any toxic fragment in the toxic fragment dataset, the concept label corresponding to the toxic fragment is recorded, and the hidden representation before the last token of the generated toxic fragment is extracted.

[0042] Specifically, in the toxicity-inducing scenario, the large language model performs autoregressive generation. It is given hints from the toxicity-inducing dataset. The token sequence generated by the model in the response is The corresponding latent space representation sequence is ( .

[0043] In the decoding step At that time, check the current token and its predecessors. Does the token match a certain toxic fragment? If satisfied Then record the toxic fragment. Corresponding concept tags And the hidden representation prior to the generation of the last constituent token of this toxic fragment. In this invention, based on the model's word segmentation mechanism and the length of the toxic corpus, the... The range of values ​​is defined as follows .

[0044] Select The reason is that this state corresponds to the decision state before the model generates toxic expressions, and therefore contains semantic signals that predict the content to be generated. This results in the concept representation dataset: .

[0045] S23: Input the extracted hidden representation into the sparse autoencoder to obtain the corresponding sparse activation vector, count the activation frequency of each sparse feature dimension under different concept labels, establish the mapping relationship between each toxicity concept and the corresponding sparse feature dimension, and form a mapping set.

[0046] Specifically, the extracted hidden representations Input the sparse autoencoder to obtain the corresponding sparse activation vector. Based on the sparse activation vector obtained from each input, the activation frequency and intensity of each sparse feature dimension under different concept labels are statistically analyzed to locate the feature dimensions related to different toxicity concepts. A mapping relationship between each toxicity concept and its corresponding sparse feature dimension is established to obtain the feature dimension set corresponding to each toxicity concept, forming a mapping set.

[0047] Based on the feature dimension set corresponding to each toxicity concept, toxicity detection can be performed by observing the activation patterns of the located features during the content generation process, and fine-grained intervention can be applied only to sparse latent features related to the toxicity concept, thereby better maintaining the natural language distribution in the model space.

[0048] In the autoregressive generation process, this invention applies to each decoding step. Both are based on hidden representations Observe the corresponding sparse activation vectors In this embodiment, the concept of toxicity is... The set of feature dimensions corresponding to it in the mapping set is toxicity concept In the current decoding step The activation score is calculated according to the following formula: ; In the formula, Toxicity concept Activation score, The concept of toxicity in mapping sets The corresponding set of feature dimensions, sparse activation vectors Dimensions activation value, This indicates the decoding step.

[0049] That is, from sparse activation vectors Extracting the feature dimension set The activation values ​​across all dimensions are used, and the maximum value is taken as the activation score for the toxicity concept. This score reflects the activation strength of the toxicity concept semantics in the current decoding step.

[0050] Step 3: Compare the activation scores of each toxic concept with a preset threshold. If the activation scores of all toxic concepts do not exceed the preset threshold, it is determined that no toxic semantics were detected in the current decoding step, and the process jumps to step 5. Otherwise, add the toxic concepts with activation scores greater than the preset threshold to the set of activated concepts, and select the toxic concept with the highest activation score as the dominant toxic concept.

[0051] Unlike static representation editing methods that continuously intervene throughout the generation process, this invention uses dynamic detection based on concept-level sparse activation. To filter out false activations that may be caused by syntactic structure or local noise, a threshold mechanism is introduced. Only when the activation score of a toxic concept exceeds the threshold is the toxic concept considered activated in the current step.

[0052] Specifically, the activation scores of each toxicity concept are compared with preset thresholds. A comparison is then made. If the activation scores corresponding to all toxicity concepts do not exceed the preset threshold, i.e. If no toxic semantics are detected in the current decoding step, no intervention is required, and the process proceeds to step 5.

[0053] Otherwise, add toxicity concepts with activation scores greater than a preset threshold to the set of activated concepts. .like A non-empty value indicates that at least one toxic concept has been detected as activated in the current decoding step. When the set of activated concepts contains multiple toxic concepts, [the value is determined by the context]. The toxicity concept with the highest activation score was selected as the dominant toxicity concept.

[0054] In this embodiment, a preset threshold is used. The same value is used for all toxicity concepts. In a preferred embodiment of the invention, a preset threshold is used. The threshold was determined by adjusting it on the validation set while keeping the intervention intensity constant, and by comprehensively evaluating the toxicity score and the generated perplexity. See also... Figure 4 , Figure 4 The figure shown is an experimental result graph showing the effect of the activated filter threshold on the toxicity score and generation fluency.

[0055] like Figure 4 As shown, in the GPT2-small model, a threshold of approximately 0.4 achieves a good balance between toxicity suppression and language fluency. Lower thresholds fail to effectively filter noise features triggered by syntactic structure; although the toxicity score generated by the model is lower than the original model, it significantly increases perplexity (PPL), leading to a marked decrease in language fluency. Higher thresholds better maintain the fluency of the generated text, keeping PPL at a low level, but their toxicity suppression effect is relatively weakened.

[0056] Step 4: Based on the pre-determined sparse detoxification direction corresponding to each toxicity concept, determine the sparse detoxification direction of the dominant toxicity concept, and apply a directional inhibition operation to the sparse activation vector according to the sparse detoxification direction of the dominant toxicity concept to obtain the intervened sparse activation vector.

[0057] In this embodiment, precise intervention on different toxicity concepts is achieved by constructing a specific detoxification direction for each toxicity concept. Specifically, the process of determining the sparse detoxification direction corresponding to each toxicity concept includes the following steps: S31: For each toxicity concept, collect multiple sample pairs formed by toxic and non-toxic samples, input the sample pairs into a large language model to extract the corresponding toxic hidden representation and non-toxic hidden representation, and input the toxic hidden representation and non-toxic hidden representation into a sparse autoencoder to obtain the corresponding toxic sparse activation vector and non-toxic sparse activation vector.

[0058] Specifically, the concept of toxicity Multiple sample pairs consisting of toxic and non-toxic samples are collected. For each sample pair, the toxic and non-toxic samples are input into a large language model to extract the corresponding toxic and non-toxic hidden representations. These toxic and non-toxic hidden representations are then input into a sparse autoencoder to obtain the corresponding toxic sparse activation vectors. With non-toxic sparse activation vectors .

[0059] S32: Calculate the difference vector for each pair of samples based on the toxic sparse activation vector and the non-toxic sparse activation vector for each pair of samples.

[0060] For the For a sample, the difference vector Calculate according to the following formula: .

[0061] This difference vector represents the direction and magnitude of change in each feature dimension in the sparse representation space when shifting from "non-toxic semantics" to "specific toxic semantics".

[0062] concept of toxicity Collect the difference vectors of all sample pairs under this toxicity concept to form a difference vector set. ,in This represents the total number of sample pairs for this toxicity concept.

[0063] S33: Perform principal component analysis on the set of all difference vectors under each toxicity concept to determine the corresponding covariance matrix. Extract the eigenvector corresponding to the largest eigenvalue of the covariance matrix as the first principal component and determine the first principal component as the initial guiding direction of the corresponding toxicity concept.

[0064] Specifically, regarding the concept of toxicity The set of difference vectors First, calculate the empirical mean: .

[0065] Then calculate the covariance matrix: .

[0066] The covariance matrix This is used to measure the synergistic changes among various sparse feature dimensions when distinguishing between toxic and non-toxic activations.

[0067] The eigenvector corresponding to the largest eigenvalue of the covariance matrix is ​​extracted as the first principal component, i.e.: The first principal component This toxicity concept was identified. The initial guiding direction. Statistically, this direction represents the axis with the largest variance in the difference between toxic and non-toxic activations under this concept, i.e., the trajectory of the most significant potential characteristic changes that distinguish this toxicity concept.

[0068] S34: Based on the feature dimension set corresponding to each toxicity concept in the mapping set, the initial guiding direction corresponding to each toxicity concept is sparsified to obtain the sparse detoxification direction of each toxicity concept.

[0069] In this embodiment, the sparsity processing is specifically as follows: for the initial guiding direction of each toxicity concept, retain the dimension values ​​of the feature dimension set corresponding to the toxicity concept, and set the remaining dimensions to zero.

[0070] Regarding the concept of toxicity The direction of sparse detoxification is represented as: .

[0071] The resulting sparse detoxification direction is a highly sparse vector with non-zero values ​​only on a few pre-defined feature dimensions that are strongly correlated with the toxicity concept. This sparsity ensures that the intervention only applies to the feature subspace truly related to the toxicity concept, avoiding interference with other semantic dimensions.

[0072] In this embodiment, the sparse activation vector after intervention is calculated according to the following formula: ; In the formula, The sparse activation vector after intervention. For sparse activation vectors, The sparse detoxification approach, which is the dominant concept of toxicity, This is a scaling factor used to control the intensity of the detoxification intervention.

[0073] The optimal value can be determined by traversing and searching on the validation set. In a preferred embodiment of the present invention, the values ​​for the three models GPT2-small, Qwen3-1.7B, and Gemma2-2B are 5, 9, and 5, respectively.

[0074] Sparse detoxification direction Only in the set of feature dimensions corresponding to this toxicity concept The resulting subspace contains non-zero values, while the other dimensions are zero. Therefore, the intervention operation only makes local adjustments in the sparse subspace related to the toxicity concept, without affecting other irrelevant semantic features. Through this concept-level guidance mechanism, this invention can suppress the generation of harmful semantics while minimizing perturbations to irrelevant semantics, thereby better maintaining the fluency of the generated text and the model's generality.

[0075] Step 5: Input the sparse activation vector or the intervened sparse activation vector into the sparse autoencoder for reconstruction processing to obtain the adjusted hidden representation. Then, feed the adjusted hidden representation back to the large language model to continue subsequent token generation.

[0076] Specifically, if no toxicity is detected, the sparse activation vector that was not intervened in step 3 will be... The decoder of the input sparse autoencoder is reconstructed to obtain the adjusted hidden representation; if toxicity is detected and intervention is performed, the sparse activation vector after intervention in step 4 is... The decoder of the input sparse autoencoder is reconstructed to obtain the adjusted hidden representation. The adjusted hidden representation is fed back to the large language model to replace the original hidden representation, which is then used by subsequent network layers of the large language model, such as feedforward neural network layers and layer normalization layers, to complete the final output probability distribution calculation and token sampling for the current token.

[0077] Repeat steps 1 to 5 above until the large language model generates an end-of-symbol (EOS) or reaches the preset maximum generation length, thus completing the generation of all tokens and obtaining the output text after toxicity suppression.

[0078] This invention presents a large language model toxicity removal method based on concept guidance within a sparse representation space. It combines a mapping mechanism between toxic concepts and sparse features with a fine-grained intervention strategy guided by concepts, achieving efficient and accurate inference control of toxic semantics. When a target toxic concept is detected as being triggered, a targeted suppression operation is applied only to the sparse feature subspace corresponding to that concept. This achieves localized and selective representation adjustment, effectively reducing interference with irrelevant semantic dimensions and ensuring the naturalness and semantic consistency of the generated text.

[0079] Furthermore, the effectiveness of the toxicity removal method for large language models based on concept guidance in sparse representation space of the present invention is verified and illustrated through simulation experiments.

[0080] Large-scale experiments were conducted on three large language models (GPT-2-small, Qwen3-1.7B, and Gemma-2B) for validation. The experiments were carried out on a challenging subset of the RealToxicityPrompts dataset, and the baselines for comparison included Destein, ProFS, Prompting, and the SAE-Detoxify method, which is also based on SAE for detoxification. The experimental results are shown in Table 1.

[0081] Table 1

[0082] Experimental results show that the toxicity score of the method of the present invention can be reduced by up to 62.69%. At the same time, in terms of language fluency (PPL), the method of the present invention achieves the best or second best results on all three models.

[0083] In addition to its superior detoxification effect and high language fluency, the method of this invention also maintains strong inference efficiency, with a running speed almost comparable to the original large language model. Furthermore, the method of this invention performs well in preserving the model's original capabilities, with minimal impact on general task performance. On the seven general capability evaluation tasks provided by EleutherAI LM Harness, the method of this invention achieved average accuracies of 39.22%, 59.76%, and 63.84% on GPT2-small, Qwen3-1.7B, and Gemma2-2B, respectively, indicating that the method of this invention can effectively reduce toxicity while retaining the model's general capabilities.

[0084] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that an article or device comprising a list of elements includes not only those elements but also other elements not expressly listed. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device comprising said element. Terms such as "connected" or "linked" are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect. The orientations or positional relationships indicated by terms such as "upper," "lower," "left," and "right" are based on the orientations or positional relationships shown in the accompanying drawings and are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as limiting the invention.

[0085] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0086] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A method for removing toxicity from a large language model based on concept guidance within a sparse representation space, characterized in that, include: Step 1: In the current decoding step, obtain the hidden representation of the current token by the large language model, and input the hidden representation into the pre-trained sparse autoencoder for encoding to obtain the corresponding sparse activation vector. Step 2: Calculate the activation score corresponding to each toxicity concept based on the sparse activation vector and the pre-constructed mapping set between toxicity concepts and sparse feature dimensions; Step 3: Compare the activation scores of each toxicity concept with a preset threshold. If the activation scores of all toxicity concepts do not exceed the preset threshold, it is determined that no toxic semantics were detected in the current decoding step, and the process proceeds to step 5. Otherwise, toxicity concepts with activation scores greater than the preset threshold are added to the set of activated concepts, and the toxicity concept with the highest activation score is selected as the dominant toxicity concept. Step 4: Based on the pre-determined sparse detoxification direction corresponding to each toxicity concept, determine the sparse detoxification direction of the dominant toxicity concept, and apply a directional inhibition operation to the sparse activation vector according to the sparse detoxification direction of the dominant toxicity concept to obtain the intervened sparse activation vector. Step 5: Input the sparse activation vector or the intervened sparse activation vector into the sparse autoencoder for reconstruction processing to obtain the adjusted hidden representation. Then, feed the adjusted hidden representation back to the large language model to continue subsequent token generation.

2. The method for removing toxicity from large language models based on concept guidance within a sparse representation space according to claim 1, characterized in that, The training process of the sparse autoencoder includes: S11: Input multiple paired labeled toxic and non-toxic responses into the large language model, extract the corresponding hidden representations, and form a training dataset; S12: Using the training dataset as the target, the sparse autoencoder is trained with the joint loss function consisting of reconstruction error and sparse regularization term, so that the sparse autoencoder learns sparse feature representations with discriminative ability for toxic semantics during the process of encoding hidden representations into sparse activation vectors; wherein, the joint loss function is expressed as: in, For the original hidden representation, For the reconstruction of hidden representations by sparse autoencoders, For sparse activation vectors, This represents the sparsity weighting coefficient.

3. The method for removing toxicity from large language models based on concept guidance within a sparse representation space according to claim 1, characterized in that, The process of constructing the mapping set between the toxicity concept and the sparse feature dimension includes: S21: Use an auxiliary large language model to extract text fragments containing toxic semantics from toxic responses, and perform concept annotation on each text fragment according to a predefined set of toxic concepts to construct a concept-annotated toxic fragment dataset; S22: In the toxicity-induced scenario, the large language model performs autoregressive generation. When the generated token sequence matches any toxic fragment in the toxic fragment dataset, the concept label corresponding to the toxic fragment is recorded, and the hidden representation before the last token of the toxic fragment is extracted. S23: Input the extracted hidden representation into the sparse autoencoder to obtain the corresponding sparse activation vector, count the activation frequency of each sparse feature dimension under different concept labels, establish the mapping relationship between each toxicity concept and the corresponding sparse feature dimension, and form the mapping set.

4. The method for removing toxicity from large language models based on concept guidance within a sparse representation space according to claim 3, characterized in that, The predefined set of toxicity concepts includes: Obscene, Threat, Insult, and Identity_Hate.

5. The method for removing toxicity from large language models based on concept guidance within a sparse representation space according to claim 1, characterized in that, The activation score for each toxicity concept is calculated using the following formula: ； In the formula, Toxicity concept Activation score, The concept of toxicity in mapping sets The corresponding set of feature dimensions, sparse activation vectors Dimensions activation value, This indicates the decoding step.

6. The method for removing toxicity from large language models based on concept guidance within a sparse representation space according to claim 1, characterized in that, The process of determining the sparse detoxification direction corresponding to each toxicity concept includes: S31: For each toxicity concept, collect multiple sample pairs formed by toxic and non-toxic samples, input the sample pairs into the large language model to extract the corresponding toxic hidden representation and non-toxic hidden representation, and input the toxic hidden representation and non-toxic hidden representation into the sparse autoencoder to encode the corresponding toxic sparse activation vector and non-toxic sparse activation vector. S32: Calculate the difference vector for each pair of samples based on the toxic sparse activation vector and the non-toxic sparse activation vector for each pair of samples. S33: Perform principal component analysis on all difference vector sets under each toxicity concept to determine the corresponding covariance matrix, extract the eigenvector corresponding to the largest eigenvalue of the covariance matrix as the first principal component, and determine the first principal component as the initial guiding direction of the corresponding toxicity concept. S34: Based on the feature dimension set corresponding to each toxicity concept in the mapping set, the initial guiding direction corresponding to each toxicity concept is sparsified to obtain the sparse detoxification direction of each toxicity concept.

7. The method for removing toxicity from large language models based on concept guidance within a sparse representation space according to claim 6, characterized in that, The sparsification process includes: for each toxicity concept, retaining the dimension values ​​of the feature dimension set corresponding to the toxicity concept in the initial guiding direction, and setting the remaining dimensions to zero.

8. The method for removing toxicity from large language models based on concept guidance within a sparse representation space according to claim 1, characterized in that, The sparse activation vector after intervention is calculated according to the following formula: ； In the formula, The sparse activation vector after intervention. For sparse activation vectors, This is the scaling factor. The sparse detoxification direction is the dominant concept of toxicity.

9. The method for removing toxicity from a large language model based on concept guidance within a sparse representation space according to claim 1, characterized in that, The latent space dimension of the sparse autoencoder is greater than that of the latent space dimension of the large language model. The sparse autoencoder decomposes the dense hidden representation of the large language model into sparse representations through an overcomplete dictionary. The encoder of the sparse autoencoder uses ReLU or JumpReLU activation functions to achieve sparse activation.

Images