A data debiasing and alignment enhancement method for large language models, electronic equipment and storage medium
By performing text decomposition, sentiment annotation, and model construction on large language models, the problem of difficult-to-eliminate bias in the training of large language models is solved, and efficient bias recognition and neutral output are achieved.
Patent Information
- Application Number
- CN202511465889.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-14
- Publication Date
- 2026-05-15
- Estimated Expiration
- 2045-10-14
AI Technical Summary
Existing large language models suffer from biases during training, resulting in unfair and biased output. Existing methods struggle to effectively detect and eliminate multi-dimensional biases, and are difficult to optimize during training and lack real-time review during use.
By collecting texts from multiple domains, performing sentence-by-sentence decomposition and sentiment annotation, dividing the target conflict groups, constructing neutral dialogue models and bias models, and using template matching and multi-model voting group decision-making for training and real-time review.
It achieves efficient identification and elimination of bias, optimizes the training process, provides a real-time review mechanism, and improves the neutrality and accuracy of model output.
Smart Images
Figure CN121352002B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of generative artificial intelligence technology, and in particular to a data debiasing and alignment enhancement method, electronic device, and storage medium for large language models. Background Technology
[0002] In recent years, although the performance of large language models has been significantly improved with the training of large amounts of data, enabling them to perform tasks in multiple scenarios such as natural language processing, question answering, and text generation, they generally suffer from bias due to reasons such as inadequate text screening. Due to the unbalanced types of training data, or the presence of non-objective negative evaluations based on socio-cultural factors, individual cognition, and experience, the models learn and solidify these biases during training, resulting in unfair and biased expressions in the output content.
[0003] To address this issue, existing technologies primarily focus on: at the data and detection level, reducing bias caused by data imbalance by collecting more diverse data, or performing bias detection on the data to analyze bias characteristics. However, the latter is difficult to fully understand and eliminate all biases due to their multi-dimensional and complex nature. At the intervention and training level, real-time intervention strategies in the generation process rely on users actively providing anti-biased examples, which limits application scenarios and can only superficially correct the output, failing to eradicate the model's inherent bias. While training methods based on human preference alignment, such as direct preference optimization, forcefully expand the difference in generation probabilities between biased and neutral responses by constructing bias-neutral response pairs, driving the model towards unbiased output, the lack of quantitative assessment of bias intensity prevents precise optimization for biases of varying severity. Furthermore, focusing solely on maximizing the probability difference may lead to a simultaneous increase in the generation probabilities of neutral and biased responses, severely weakening the effectiveness of the debiasing system. Summary of the Invention
[0004] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a data bias removal and alignment enhancement method, electronic device, and storage medium for large language models, which solves the problems of existing methods such as complex multi-dimensional crossover in data bias detection, difficulty in optimizing question-answer pairs during training, and lack of real-time review during use.
[0005] To achieve the above objectives, the present invention provides the following solution:
[0006] A data debiasing and alignment enhancement method for large language models includes:
[0007] Collect text content covering multiple fields to obtain general text;
[0008] A structured descriptive template is set, and the general text is broken down into individual sentences to obtain individual sentence texts;
[0009] The decomposed single-sentence text is matched with the structured description template to obtain the extraction template;
[0010] Based on the extraction template, sentiment annotation is performed on the single sentence text to obtain single sentence sentiment tags; the single sentence sentiment tags include: sentiment text and no sentiment text.
[0011] The single-sentence texts in the emotional texts that have potential contrasting relationships between objects are divided into object conflict groups;
[0012] The percentage of each object in each category of sentiment description in each object conflict group is calculated to obtain the single object percentage value. The single sentence text corresponding to the object conflict group whose single object percentage value does not meet the preset allowable deviation threshold is set as biased text, and the single sentence text corresponding to the object conflict group whose single object percentage value meets the preset allowable deviation threshold is set as neutral text.
[0013] By fusing the emotionless text and the neutral text, a pre-biased text is obtained. The pre-biased text is then used to train a pre-defined large language model to obtain a neutral dialogue model.
[0014] The bias model is obtained by training the preset large language model using the biased text and the emotionless text;
[0015] Receive user input text, and simultaneously input the user input text into the neutral dialogue model and the bias model to obtain a first answer and a second answer;
[0016] The first and second answers are calculated using a pre-built review formula to obtain the dual-model output difference.
[0017] When the difference between the outputs of the two models is less than the difference standard, return to the step "Receive user input text, and synchronously input the user input text into the neutral dialogue model and the bias model to obtain the first answer and the second answer";
[0018] When the difference between the outputs of the dual models is greater than or equal to the difference standard, the first answer is output, and a neutral answer is obtained.
[0019] Preferably, a structured descriptive template is set, and the general text is broken down into single sentences to obtain single-sentence text, including:
[0020] The general text is split using the HanLP sentence segmenter based on a multi-dimensional splitting mechanism to obtain the original split text;
[0021] The integrity of the original split text is determined. When the integrity is that the sentence components are incomplete, the current original split text is combined with the adjacent original split texts to obtain the single sentence text.
[0022] Preferably, the decomposed single-sentence text is matched with the structured description template to obtain an extraction template, including:
[0023] Define the structured description template; the structured description template includes: subject object + sentiment adjective, subject object + behavior description + sentiment adjective, subject object + comparison object + domain type + degree word, subject object + scene limitation + comparison object + degree word;
[0024] The single-sentence text is extracted according to the set element positioning rules to obtain core elements; the core elements include: subject object, comparison object, sentiment adjectives, behavior description, domain type, scene limitation, degree words;
[0025] The structured description template is matched based on the types of elements within the core elements of a single sentence text to obtain the extracted template.
[0026] Preferably, sentiment annotation is performed on the single-sentence text according to the extraction template to obtain single-sentence sentiment tags, including:
[0027] The single sentence text is tagged with part-of-speech tags using a part-of-speech tagging tool, and the sentiment score of each word in the tagged single sentence text is obtained using the SentiWordNet tool to obtain the single sentence sentiment score.
[0028] The single-sentence text in which the number of subject objects or emotional adjectives is zero is set as the emotionless text;
[0029] If the number of subject objects or emotional adjectives is not zero, and the emotional score of the single sentence belongs to the neutral score range, then the single sentence text is set as emotionless text; otherwise, the single sentence text is set as emotional text.
[0030] Preferably, the percentage of each object in each category of sentiment description within each object conflict group is calculated to obtain a single object percentage value. The single sentence text corresponding to the object conflict group whose single object percentage value does not meet a preset allowable deviation threshold is set as biased text, and the single sentence text corresponding to the object conflict group whose single object percentage value meets the preset allowable deviation threshold is set as neutral text. This includes:
[0031] Extract the maximum value of the percentage of a single object within the same conflict group to obtain the maximum percentage; the expression for the maximum percentage is: ;in, For the first The first group of objects in the conflict group The maximum percentage of each emotional description; This is the function for extracting the maximum value. For the first The first group of objects in the conflict group The object in the first The percentage of a single object in a sentiment description;
[0032] The relative proportion is obtained by calculating the ratio of the proportion of each of the remaining single objects in the same object conflict group to the maximum proportion; the expression for the relative proportion is:
[0033] ;in, For the first The relative proportion of each object; The object corresponding to the maximum percentage;
[0034] The single-sentence text corresponding to the object conflict group whose relative proportion is greater than or equal to the preset allowable deviation threshold is set as the neutral text, and the single-sentence text corresponding to the object conflict group whose relative proportion is less than the preset allowable deviation threshold is set as the biased text.
[0035] Preferably, the unemotional text and the neutral text are fused to obtain pre-biased text, and the pre-biased text is used to train a preset large language model to obtain a neutral dialogue model, including:
[0036] The pre-biased text is input into an LLMs model to generate question-answer pairs, resulting in the original question-answer pairs.
[0037] The biased answer pairs in the original question-answer pairs are eliminated using a multi-model voting group decision-making method to obtain neutral question-answer pairs; the biased answer pairs have more than 50% of the model voting results indicating bias in the multi-model voting group decision-making process.
[0038] The pre-biased text is masked using autoregressive language modeling and masked language modeling to obtain a pre-training task.
[0039] The pre-set large language model is trained using the neutral question-answer pairs and the pre-training task according to the supervised fine-tuning mechanism, and the parameters of the pre-set large language model are iteratively optimized using the AdamW optimizer to obtain the neutral dialogue model.
[0040] Preferably, the bias model is obtained by training the preset large language model using the biased text and the emotionless text, including:
[0041] The biased text and the emotionless text are masked using autoregressive language modeling and masked language modeling to obtain a pre-training task.
[0042] The bias model is obtained by training the pre-training task using the unsupervised mechanism to obtain the bias model.
[0043] Preferably, the pre-built review formula is:
[0044] ;in, Output the difference degree for the two models; , These are the weights for semantic similarity differences and sentiment difference factors, respectively. , These are the semantic embedding vectors of the first answer and the second answer, respectively; , These are the overall sentiment scores for the first and second responses, respectively.
[0045] Preferably, an electronic device includes: at least one processor and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the aforementioned data debiasing and alignment enhancement method for a large language model.
[0046] Preferably, a non-transitory computer-readable storage medium stores computer instructions for causing a computer to execute the aforementioned data debiasing and alignment enhancement method for a large language model.
[0047] The present invention discloses the following technical effects:
[0048] This invention provides a data bias removal and alignment enhancement method, electronic device, and storage medium for large language models. By using template matching, sentiment annotation, and conflict group partitioning, it solves the problem of complex and multi-dimensional data bias detection in existing methods, and realizes the transformation of complex bias detection into object and sentiment comparison statistical analysis. By constructing a neutral dialogue model and a bias model, it solves the problems of difficulty in optimizing question-answer pairs during training and the lack of real-time review during use, and realizes automatic generation of erroneous references and optimization of iterative direction during model training. Attached Figure Description
[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0050] Figure 1 This is a schematic diagram of the data debiasing and alignment enhancement process for a large language model provided in an embodiment of the present invention;
[0051] Figure 2 This is a flowchart of data debiasing and alignment enhancement provided in an embodiment of the present invention;
[0052] Figure 3 This is a schematic diagram of the sentiment annotation process provided in an embodiment of the present invention;
[0053] Figure 4 This is a schematic diagram of the biased text and neutral text judgment process provided in an embodiment of the present invention;
[0054] Figure 5 This is a schematic diagram illustrating the training process of the bias model and the neutral dialogue model provided in the embodiments of the present invention. Detailed Implementation
[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0056] The purpose of this invention is to provide a data bias removal and alignment enhancement method, electronic device, and storage medium for large language models, which solves the problems of existing methods such as complex multi-dimensional crossover in data bias detection, difficulty in optimizing question-answer pairs during training, and lack of real-time review during use.
[0057] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0058] Figure 1 This is a schematic diagram of the data bias removal and alignment enhancement process for a large language model provided in an embodiment of the present invention. Figure 2 The flowchart for data debiasing and alignment enhancement provided in the embodiments of the present invention is as follows: Figure 1 and Figure 2 As shown, this invention provides a data debiasing and alignment enhancement method for large language models, comprising:
[0059] Step 100: Collect text content covering multiple fields to obtain general text;
[0060] Step 200: Set a structured description template, and decompose the general text into single sentences to obtain single-sentence text;
[0061] Step 300: Match the decomposed single-sentence text with the structured description template to obtain the extraction template;
[0062] Step 400: Perform sentiment annotation on the single sentence text according to the extraction template to obtain single sentence sentiment tags; the single sentence sentiment tags include: sentiment text and no sentiment text;
[0063] Step 500: Divide the single-sentence texts in the emotional text into object conflict groups where there is a potential contrast relationship between objects;
[0064] Step 600: Calculate the proportion of each object in each category of sentiment description in each object conflict group to obtain the single object proportion value. Set the single sentence text corresponding to the object conflict group whose single object proportion value does not meet the preset allowable deviation threshold as biased text, and set the single sentence text corresponding to the object conflict group whose single object proportion value meets the preset allowable deviation threshold as neutral text.
[0065] Step 700: Fuse the emotionless text and the neutral text to obtain the pre-biased text, and use the pre-biased text to train the preset large language model to obtain the neutral dialogue model;
[0066] Step 800: Train the preset large language model using the biased text and the emotionless text to obtain the bias model;
[0067] Step 900: Receive user input text, and synchronously input the user input text into the neutral dialogue model and the bias model to obtain the first answer and the second answer;
[0068] Step 1000: Calculate the difference between the first and second answers using a pre-built review formula to obtain the dual-model output difference.
[0069] Step 1100: When the difference between the outputs of the two models is less than the difference standard, return to the step "Receive user input text, and synchronously input the user input text into the neutral dialogue model and the bias model to obtain the first answer and the second answer";
[0070] Step 1200: When the difference between the outputs of the dual models is greater than or equal to the difference standard, the first answer is output to obtain a neutral answer.
[0071] Preferably, a structured descriptive template is set, and the general text is broken down into single sentences to obtain single-sentence text, including:
[0072] The general text is split using the HanLP sentence segmenter based on a multi-dimensional splitting mechanism to obtain the original split text;
[0073] The integrity of the original split text is determined. When the integrity is that the sentence components are incomplete, the current original split text is combined with the adjacent original split texts to obtain the single sentence text.
[0074] Further, the decomposed single-sentence text is matched with the structured description template to obtain the extraction template, including:
[0075] Define the structured description template; the structured description template includes: subject object + sentiment adjective, subject object + behavior description + sentiment adjective, subject object + comparison object + domain type + degree word, subject object + scene limitation + comparison object + degree word;
[0076] The single-sentence text is extracted according to the set element positioning rules to obtain core elements; the core elements include: subject object, comparison object, sentiment adjectives, behavior description, domain type, scene limitation, degree words;
[0077] The structured description template is matched based on the types of elements within the core elements of a single sentence text to obtain the extracted template.
[0078] refer to Figure 3 Based on the extraction template, sentiment annotation is performed on the single-sentence text to obtain single-sentence sentiment tags, including:
[0079] Step 401: Use a part-of-speech tagging tool to perform part-of-speech tagging on the single sentence text, and use SentiWordNet tool to obtain the sentiment score of each word in the single sentence text after part-of-speech tagging, so as to obtain the single sentence sentiment score;
[0080] Step 402: Set the single-sentence text with zero subject objects or emotional adjectives as the emotionless text;
[0081] Step 403: When the number of subject objects or sentiment adjectives is not zero, if the sentiment score of the single sentence belongs to the neutral score range, then the single sentence text is set as emotionless text; otherwise, the single sentence text is set as sentimental text.
[0082] refer to Figure 4The percentage of each object in each category of sentiment description within each object conflict group is calculated to obtain a single object percentage value. The single sentence text corresponding to the object conflict group whose single object percentage value does not meet a preset allowable deviation threshold is set as biased text, and the single sentence text corresponding to the object conflict group whose single object percentage value meets the preset allowable deviation threshold is set as neutral text. This includes:
[0083] Step 601: Extract the maximum value of the single object proportion value in the same object conflict group to obtain the maximum proportion; the expression for the maximum proportion is: ;in, For the first The first group of objects in the conflict group The maximum percentage of each emotional description; This is the function for extracting the maximum value. For the first The first group of objects in the conflict group The object in the first The percentage of a single object in a sentiment description;
[0084] Step 602: Calculate the ratio of the percentage of each of the remaining single objects in the same object conflict group to the maximum percentage to obtain the relative percentage; the expression for the relative percentage is:
[0085] ;in, For the first The relative proportion of each object; The object corresponding to the maximum percentage;
[0086] Step 603: Set the single sentence text corresponding to the object conflict group whose relative proportion is greater than or equal to the preset allowable deviation threshold as the neutral text, and set the single sentence text corresponding to the object conflict group whose relative proportion is less than the preset allowable deviation threshold as the biased text.
[0087] refer to Figure 5 The unemotional text and the neutral text are fused to obtain pre-biased text. The pre-biased text is then used to train a pre-defined large language model to obtain a neutral dialogue model, including:
[0088] The pre-biased text is input into an LLMs model to generate question-answer pairs, resulting in the original question-answer pairs.
[0089] The biased answer pairs in the original question-answer pairs are eliminated using a multi-model voting group decision-making method to obtain neutral question-answer pairs; the biased answer pairs have more than 50% of the model voting results indicating bias in the multi-model voting group decision-making process.
[0090] The pre-biased text is masked using autoregressive language modeling and masked language modeling to obtain a pre-training task.
[0091] The pre-set large language model is trained using the neutral question-answer pairs and the pre-training task according to the supervised fine-tuning mechanism, and the parameters of the pre-set large language model are iteratively optimized using the AdamW optimizer to obtain the neutral dialogue model.
[0092] refer to Figure 5 The bias model is obtained by training the pre-defined large language model using the biased text and the emotionless text, including:
[0093] The biased text and the emotionless text are masked using autoregressive language modeling and masked language modeling to obtain a pre-training task.
[0094] The bias model is obtained by training the pre-training task using the unsupervised mechanism to obtain the bias model.
[0095] Furthermore, the pre-built review formula is as follows:
[0096] ;in, Output the difference degree for the two models; , These are the weights for semantic similarity differences and sentiment difference factors, respectively. , These are the semantic embedding vectors of the first answer and the second answer, respectively; , These are the overall sentiment scores for the first and second responses, respectively.
[0097] Specifically, a large amount of general text is collected and broken down into individual sentences; the main subject and the descriptive part describing the main subject are extracted from each sentence; the extracted descriptive parts are sentiment-labeled; based on the labeling, the sentences are divided into two parts: sentiment-laden text and non-sentiment-laden text; main subjects with potential opposing or contrasting relationships in the sentiment-laden text are grouped into the same object conflict group; the proportion of each main subject in different sentiment descriptions within the same object conflict group is statistically analyzed; if the proportions of different subjects in the same group differ significantly in the same sentiment description, it indicates a "bias" in that sentiment description, and the corresponding text is a non-neutral descriptive text; non-neutral descriptive texts are removed from the original text set to obtain neutral texts.
[0098] Preferably, the multi-dimensional segmentation mechanism includes grammatical structure dimension, semantic coherence dimension, and punctuation standardization dimension. Compared with traditional segmenters, HanLP sentence segmenter, with its adaptability to complex Chinese sentence structures, can identify the core predicate and its governing components through dependency parsing, reducing the segmentation error of ambiguous sentences. The multi-dimensional segmentation mechanism is imported into the HanLP sentence segmenter, and the text is initially segmented according to the priority of grammar, semantics, and punctuation. Using HanLP's part-of-speech tagging tool, the presence of a core subject-predicate structure is detected. A semantic similarity model is used to determine whether a segment requires context for understanding. When the original segmented text has grammatical incompleteness or semantic non-independence, it is combined with neighboring original segmented texts. The combination must follow the principle of minimum scope supplementation: that is, it should be combined with the previous or next text first. If the above problems still exist, the combination domain is expanded sequentially. If combining three texts still does not meet the requirements, the text is judged to have a semantic break, and the corresponding text is removed.
[0099] Furthermore, this embodiment sets up various structured adjective templates based on conventional sentence layouts, including but not limited to: subject object + sentiment adjective, subject object + behavior description + sentiment adjective, subject object + comparison object + domain type + degree word, and subject object + scene limitation + comparison object + degree word. The correspondence between different elements and attribute features is as follows: subject object or comparison object corresponds to noun or noun phrase, sentiment adjective corresponds to adjective, behavior description corresponds to verb or verb phrase, domain type corresponds to noun or noun phrase, scene limitation corresponds to prepositional phrase or noun phrase, and degree word corresponds to adverb. Natural language processing tools such as HanLP and spaCy are used to identify word attributes through part-of-speech tagging, identify logical relationships between words through dependency parsing, and match structured adjective templates according to the types of elements within the core elements of a single sentence text.
[0100] Preferably, SentiWordNet is a sentiment lexicon based on WordNet, retaining only the words corresponding to the core elements in a single sentence and removing function words without sentiment. If there are polysemous words in the core elements, disambiguation is required based on elements such as scene limitation and domain type. SentiWordNet is used to perform equal and weighted calculations on the core sentiment-related words in a single sentence to obtain the sentiment score of the single sentence. If the number of subject objects or the number of sentiment adjectives in the core elements are zero, it is determined that the core elements are missing, and the corresponding text is classified as sentimentless text. Based on the SentiWordNet score distribution and experimental verification, a neutral score interval is set. If the sentiment score of a single sentence falls within the neutral score interval, it is considered sentimentless text; otherwise, it is considered sentimental text.
[0101] Furthermore, the percentage of a single object is calculated using the frequency percentage method. The object described most frequently on a specific emotional dimension within the group is identified, and its percentage is the baseline percentage for that dimension. The extraction formula is as follows:
[0102]
[0103] In the formula For the first The first group of objects in the conflict group The largest proportion of emotional descriptions; This is the function for extracting the maximum value. For the first The first group of objects in the conflict group The object in the first The percentage of a single object in each sentiment description.
[0104] If multiple objects within a group have the same percentage and are all at the maximum value, the largest percentage is taken as the maximum percentage. The difference between the percentage of a non-maximum percentage object and the benchmark object is quantified by relative percentage. A smaller difference indicates a more balanced sentiment description; a larger difference suggests a greater likelihood of bias. The expression for relative percentage is as follows:
[0105]
[0106] In the formula For the first The relative proportion of each object; The object corresponding to the largest proportion.
[0107] Relative proportions can eliminate the influence of the total number and directly reflect the degree of imbalance, making the judgment more objective. If the relative proportion of a single sentence in a conflict group exceeds or equals a preset threshold, it is judged as neutral text; otherwise, it is judged as biased text.
[0108] Optionally, a pre-training task is constructed based on biased and unemotional text using "autoregressive language modeling + masked language modeling". A pre-defined large language model is selected and trained using an unsupervised training mode to obtain a bias model. The fused unemotional and neutral texts are then cleaned, removing fragmented text with a character length of less than 10 characters and abnormal text containing garbled characters or special symbols. The SimHash algorithm is used to deduplicate the text, setting a similarity greater than or equal to 0.95 to determine duplicate text, retaining the first occurrence of the text, resulting in clean pre-biased text. A pre-defined LLMs model is selected, and the clean pre-biased text is input into the model in a paragraph-level input-question-answer generation mode. Each text paragraph generates 2 to 3 sets of question-answer pairs, and all generated results are summarized into an original question-answer pair set. A voting model set containing 3 to 5 pre-trained models with sentiment bias detection capabilities is constructed. The original question-answer pairs are input into each voting model one by one, and the model outputs a judgment result indicating whether a pair is biased or unbiased. When more than 50% of the models determine that a question-answer pair is biased, it is marked as a biased answer pair and removed; the remaining question-answer pairs are considered neutral. A pre-training task is constructed using a combination of autoregressive language modeling and masked language modeling. A pre-set large language model is used as the base model, and a supervised fine-tuning mechanism is used for joint training. The AdamW optimizer is used for iterative parameter optimization, ultimately yielding a neutral dialogue model. During iterative training of the neutral dialogue model, the difference between the output of the bias model and the output of the current iteration step can be simultaneously incorporated into the loss function, further enhancing the neutral narrative capability during training.
[0109] Preferably, a review formula is constructed:
[0110]
[0111] In the formula, Output the difference between the two models; , These are the weights for semantic similarity differences and sentiment difference factors, respectively. , These are the semantic embedding vectors of the first and second answers, respectively. , These are the overall sentiment scores for the first and second responses, respectively.
[0112] The preprocessed user input is simultaneously fed into both the neutral dialogue model and the bias model to obtain the first answer (A) and the second answer (B). The first and second answers are then converted into semantic embedding vectors. These semantic embedding vectors are substituted into the aforementioned review formula to obtain the dual-model output difference score. When the calculated difference score is less than the difference score standard, the system returns to the neutral dialogue model for secondary generation. When the calculated difference score is greater than or equal to the difference score standard, the current first answer is output as the neutral answer.
[0113] As an optional implementation, this embodiment also provides an electronic device, including: at least one processor, and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the aforementioned data debiasing and alignment enhancement method for a large language model.
[0114] As an optional implementation, this embodiment also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the aforementioned data debiasing and alignment enhancement method for large language models.
[0115] The beneficial effects of this invention are as follows:
[0116] This invention transforms complex and cross-cutting bias detection into object and sentiment comparison statistical analysis through template matching, sentiment annotation, and conflict group division, reducing the complexity of bias removal and improving the efficiency of biased text recognition. By constructing a neutral dialogue model and a bias model, it provides erroneous references, avoiding the process of collecting a large number of people's evaluations required for traditional human preference alignment, and optimizing the training of the neutral dialogue model and the review process during operation.
[0117] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0118] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for data bias removal and alignment enhancement in large language models, characterized in that, include: Collect text content covering multiple fields to obtain general text; A structured descriptive template is set, and the general text is broken down into individual sentences to obtain individual sentence texts; The decomposed single-sentence text is matched with the structured description template to obtain the extraction template; Based on the extraction template, sentiment annotation is performed on the single sentence text to obtain single sentence sentiment tags; The single-sentence sentiment tags include: sentiment-containing text and sentimentless text; The single-sentence texts in the emotional texts that have potential contrasting relationships between objects are divided into object conflict groups; The percentage of each object in each category of sentiment description in each object conflict group is calculated to obtain the single object percentage value. The single sentence text corresponding to the object conflict group whose single object percentage value does not meet the preset allowable deviation threshold is set as biased text, and the single sentence text corresponding to the object conflict group whose single object percentage value meets the preset allowable deviation threshold is set as neutral text. By fusing the emotionless text and the neutral text, a pre-biased text is obtained. The pre-biased text is then used to train a pre-defined large language model to obtain a neutral dialogue model. The bias model is obtained by training the preset large language model using the biased text and the emotionless text; Receive user input text, and simultaneously input the user input text into the neutral dialogue model and the bias model to obtain a first answer and a second answer; The first and second answers are calculated using a pre-built review formula to obtain the dual-model output difference. When the difference between the outputs of the two models is less than the difference standard, return to the step "Receive user input text, and synchronously input the user input text into the neutral dialogue model and the bias model to obtain the first answer and the second answer"; When the difference between the outputs of the dual models is greater than or equal to the difference standard, the first answer is output, and a neutral answer is obtained.
2. The data bias removal and alignment enhancement method for large language models according to claim 1, characterized in that, A structured descriptive template is set, and the general text is broken down into single sentences to obtain single-sentence text, including: The general text is split using the HanLP sentence segmenter based on a multi-dimensional splitting mechanism to obtain the original split text; The integrity of the original split text is determined. When the integrity is that the sentence components are incomplete, the current original split text is combined with the adjacent original split texts to obtain the single sentence text.
3. The data bias removal and alignment enhancement method for large language models according to claim 1, characterized in that, The decomposed single-sentence text is matched with the structured description template to obtain the extracted template, including: Define the structured description template; the structured description template includes: subject object + sentiment adjective, subject object + behavior description + sentiment adjective, subject object + comparison object + domain type + degree word, subject object + scene limitation + comparison object + degree word; The single-sentence text is extracted according to the set element positioning rules to obtain core elements; the core elements include: subject object, comparison object, sentiment adjectives, behavior description, domain type, scene limitation, degree words; The structured description template is matched based on the types of elements within the core elements of a single sentence text to obtain the extracted template.
4. The data bias removal and alignment enhancement method for large language models according to claim 1, characterized in that, Based on the extracted template, sentiment annotation is performed on the single-sentence text to obtain single-sentence sentiment tags, including: The single sentence text is tagged with part-of-speech tags using a part-of-speech tagging tool, and the sentiment score of each word in the tagged single sentence text is obtained using the SentiWordNet tool to obtain the single sentence sentiment score. The single-sentence text in which the number of subject objects or emotional adjectives is zero is set as the emotionless text; If the number of subject objects or emotional adjectives is not zero, and the emotional score of the single sentence belongs to the neutral score range, then the single sentence text is set as emotionless text; otherwise, the single sentence text is set as emotional text.
5. The data bias removal and alignment enhancement method for large language models according to claim 1, characterized in that, The percentage of each object in each category of sentiment description within each object conflict group is calculated to obtain a single object percentage value. The single sentence text corresponding to the object conflict group whose single object percentage value does not meet a preset allowable deviation threshold is set as biased text, and the single sentence text corresponding to the object conflict group whose single object percentage value meets the preset allowable deviation threshold is set as neutral text. This includes: Extract the maximum value of the percentage of a single object within the same conflict group to obtain the maximum percentage; the expression for the maximum percentage is: ;in, For the first The first group of objects in the conflict group The maximum percentage of each emotional description; This is the function for extracting the maximum value. For the first The first group of objects in the conflict group The object in the first The percentage of a single object in a sentiment description; The relative proportion is obtained by calculating the ratio of the proportion of each of the remaining single objects in the same object conflict group to the maximum proportion; the expression for the relative proportion is: ;in, For the first The relative proportion of each object; The object corresponding to the maximum percentage; The single-sentence text corresponding to the object conflict group whose relative proportion is greater than or equal to the preset allowable deviation threshold is set as the neutral text, and the single-sentence text corresponding to the object conflict group whose relative proportion is less than the preset allowable deviation threshold is set as the biased text.
6. The data bias removal and alignment enhancement method for large language models according to claim 1, characterized in that, By fusing the unemotional text and the neutral text, a pre-biased text is obtained. This pre-biased text is then used to train a pre-defined large language model to obtain a neutral dialogue model, including: The pre-biased text is input into an LLMs model to generate question-answer pairs, resulting in the original question-answer pairs. The biased answer pairs in the original question-answer pairs are eliminated using a multi-model voting group decision-making method to obtain neutral question-answer pairs; the biased answer pairs have a bias in more than 50% of the model voting results when conducting multi-model voting group decision-making. The pre-biased text is masked using autoregressive language modeling and masked language modeling to obtain a pre-training task. The pre-defined large language model is trained using the neutral question-answer pairs and the pre-training task according to the supervised fine-tuning mechanism, and the parameters of the pre-defined large language model are iteratively optimized using the AdamW optimizer to obtain the neutral dialogue model.
7. The data bias removal and alignment enhancement method for large language models according to claim 1, characterized in that, The bias model is obtained by training the pre-defined large language model using the biased text and the emotionless text, including: The biased text and the emotionless text are masked using autoregressive language modeling and masked language modeling to obtain a pre-training task. The bias model is obtained by training the pre-training task using the unsupervised mechanism to obtain the bias model.
8. The data bias removal and alignment enhancement method for large language models according to claim 1, characterized in that, The pre-built review formula is: ;in, Output the difference degree for the two models; , These are the weights for semantic similarity differences and sentiment difference factors, respectively. , These are the semantic embedding vectors of the first answer and the second answer, respectively; , These are the overall sentiment scores for the first and second responses, respectively.
9. An electronic device, characterized in that, include: At least one processor, and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor to enable the processor to perform a data debiasing and alignment enhancement method for a large language model according to any one of claims 1 to 8.
10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute any one of claims 1 to 8, a data debiasing and alignment enhancement method for large language models.