Data depolarization and alignment enhancement method for large language model, electronic equipment and storage medium

Abstract

The invention belongs to the technical field of generative artificial intelligence, and provides a data depolarization and alignment enhancement method for a large language model, electronic equipment and a storage medium. The method comprises the steps of general text collection, single sentence disassembly, template matching, emotion labeling, object conflict group division, prejudice text and neutral text division, neutral dialogue model construction, prejudice model construction, synchronous answer generation, double-model output difference degree calculation, answer examination and neutral answer output. According to the method, through template matching, emotion labeling and conflict group division, complex cross bias detection is converted into objects, and emotion comparison statistical analysis is performed, so that the depolarization complexity is reduced, and the recognition efficiency of bias texts is improved; by constructing the neutral dialogue model and the prejudice model, error reference is provided, the process that traditional human preference alignment needs to collect a large number of personnel evaluations is avoided, and the training of the neutral dialogue model and the examination process during work are optimized.

Description

Technical Field

[0001] This invention relates to the field of generative artificial intelligence technology, and in particular to a data debiasing and alignment enhancement method, electronic device, and storage medium for large language models. Background Technology

[0002] In recent years, although the performance of large language models has been significantly improved with the training of large amounts of data, enabling them to perform tasks in multiple scenarios such as natural language processing, question answering, and text generation, they generally suffer from bias due to reasons such as inadequate text screening. Due to the unbalanced types of training data, or the presence of non-objective negative evaluations based on socio-cultural factors, individual cognition, and experience, the models learn and solidify these biases during training, resulting in unfair and biased expressions in the output content.

[0003] To address this issue, existing technologies primarily focus on: at the data and detection level, reducing bias caused by data imbalance by collecting more diverse data, or performing bias detection on the data to analyze bias characteristics. However, the latter is difficult to fully understand and eliminate all biases due to their multi-dimensional and complex nature. At the intervention and training level, real-time intervention strategies in the generation process rely on users actively providing anti-biased examples, which limits application scenarios and can only superficially correct the output, failing to eradicate the model's inherent bias. While training methods based on human preference alignment, such as direct preference optimization, forcefully expand the difference in generation probabilities between biased and neutral responses by constructing bias-neutral response pairs, driving the model towards unbiased output, the lack of quantitative assessment of bias intensity prevents precise optimization for biases of varying severity. Furthermore, focusing solely on maximizing the probability difference may lead to a simultaneous increase in the generation probabilities of neutral and biased responses, severely weakening the effectiveness of the debiasing system. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a data bias removal and alignment enhancement method, electronic device, and storage medium for large language models, which solves the problems of existing methods such as complex multi-dimensional crossover in data bias detection, difficulty in optimizing question-answer pairs during training, and lack of real-time review during use.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A data debiasing and alignment enhancement method for large language models includes:

[0007] Collect text content covering multiple fields to obtain general text;

[0008] A structured descriptive template is set, and the general text is broken down into individual sentences to obtain individual sentence texts;

[0009] The decomposed single-sentence text is matched with the structured description template to obtain the extraction template;

[0010] Based on the extraction template, sentiment annotation is performed on the single sentence text to obtain single sentence sentiment tags; the single sentence sentiment tags include: sentiment text and no sentiment text.

[0011] The single-sentence texts in the emotional texts that have potential contrasting relationships between objects are divided into object conflict groups;

[0012] The percentage of each object in each category of sentiment description in each object conflict group is calculated to obtain the single object percentage value. The single sentence text corresponding to the object conflict group whose single object percentage value does not meet the preset allowable deviation threshold is set as biased text, and the single sentence text corresponding to the object conflict group whose single object percentage value meets the preset allowable deviation threshold is set as neutral text.

[0013] By fusing the emotionless text and the neutral text, a pre-biased text is obtained. The pre-biased text is then used to train a pre-defined large language model to obtain a neutral dialogue model.

[0014] The bias model is obtained by training the preset large language model using the biased text and the emotionless text;

[0015] Receive user input text, and simultaneously input the user input text into the neutral dialogue model and the bias model to obtain a first answer and a second answer;

[0016] The first and second answers are calculated using a pre-built review formula to obtain the dual-model output difference.

[0017] When the difference between the outputs of the two models is less than the difference standard, return to the step "Receive user input text, and synchronously input the user input text into the neutral dialogue model and the bias model to obtain the first answer and the second answer";

[0018] When the difference between the outputs of the dual models is greater than or equal to the difference standard, the first answer is output, and a neutral answer is obtained.

[0019] Preferably, a structured descriptive template is set, and the general text is broken down into single sentences to obtain single-sentence text, including:

[0020] The general text is split using the HanLP sentence segmenter based on a multi-dimensional splitting mechanism to obtain the original split text;

[0021] The integrity of the original split text is determined. When the integrity is that the sentence components are incomplete, the current original split text is combined with the adjacent original split texts to obtain the single sentence text.

[0022] Preferably, the decomposed single-sentence text is matched with the structured description template to obtain an extraction template, including:

[0023] Define the structured description template; the structured description template includes: subject object + sentiment adjective, subject object + behavior description + sentiment adjective, subject object + comparison object + domain type + degree word, subject object + scene limitation + comparison object + degree word;

[0024] The single-sentence text is extracted according to the set element positioning rules to obtain core elements; the core elements include: subject object, comparison object, sentiment adjectives, behavior description, domain type, scene limitation, degree words;

[0025] The structured description template is matched based on the types of elements within the core elements of a single sentence text to obtain the extracted template.

[0026] Preferably, sentiment annotation is performed on the single-sentence text according to the extraction template to obtain single-sentence sentiment tags, including:

[0027] The single sentence text is tagged with part-of-speech tags using a part-of-speech tagging tool, and the sentiment score of each word in the tagged single sentence text is obtained using the SentiWordNet tool to obtain the single sentence sentiment score.

[0028] The single-sentence text in which the number of subject objects or emotional adjectives is zero is set as the emotionless text;

[0029] If the number of subject objects or emotional adjectives is not zero, and the emotional score of the single sentence belongs to the neutral score range, then the single sentence text is set as emotionless text; otherwise, the single sentence text is set as emotional text.

[0030] Preferably, the percentage of each object in each category of sentiment description within each object conflict group is calculated to obtain a single object percentage value. The single sentence text corresponding to the object conflict group whose single object percentage value does not meet a preset allowable deviation threshold is set as biased text, and the single sentence text corresponding to the object conflict group whose single object percentage value meets the preset allowable deviation threshold is set as neutral text. This includes:

[0031] Extract the maximum value of the percentage of a single object within the same conflict group to obtain the maximum percentage; the expression for the maximum percentage is: ;in, For the first The first group of objects in the conflict group The maximum percentage of each emotional description; This is the function for extracting the maximum value. For the first The first group of objects in the conflict group The object in the first The percentage of a single object in each emotional description;

[0032] The relative proportion is obtained by calculating the ratio of the proportion of each of the remaining single objects in the same object conflict group to the maximum proportion; the expression for the relative proportion is:

[0033] ;in, For the first The relative proportion of each object; The object corresponding to the maximum percentage;

[0034] The single-sentence text corresponding to the object conflict group whose relative proportion is greater than or equal to the preset allowable deviation threshold is set as the neutral text, and the single-sentence text corresponding to the object conflict group whose relative proportion is less than the preset allowable deviation threshold is set as the biased text.

[0035] Preferably, the unemotional text and the neutral text are fused to obtain pre-biased text, and the pre-biased text is used to train a preset large language model to obtain a neutral dialogue model, including:

[0036] The pre-biased text is input into an LLMs model to generate question-answer pairs, resulting in the original question-answer pairs.

[0037] The biased answer pairs in the original question-answer pairs are eliminated using a multi-model voting group decision-making method to obtain neutral question-answer pairs; the biased answer pairs have more than 50% of the model voting results indicating bias in the multi-model voting group decision-making process.

[0038] The pre-biased text is masked using autoregressive language modeling and masked language modeling to obtain a pre-training task.

[0039] The pre-set large language model is trained using the neutral question-answer pairs and the pre-training task according to the supervised fine-tuning mechanism, and the parameters of the pre-set large language model are iteratively optimized using the AdamW optimizer to obtain the neutral dialogue model.

[0040] Preferably, the bias model is obtained by training the preset large language model using the biased text and the emotionless text, including:

[0041] The biased text and the emotionless text are masked using autoregressive language modeling and masked language modeling to obtain a pre-training task.

[0042] The bias model is obtained by training the pre-training task using the unsupervised mechanism to obtain the bias model.

[0043] Preferably, the pre-built review formula is:

[0044] ;in, Output the difference degree for the two models; , These are the weights for semantic similarity differences and sentiment difference factors, respectively. , These are the semantic embedding vectors of the first answer and the second answer, respectively; , These are the overall sentiment scores for the first and second responses, respectively.

[0045] Preferably, an electronic device includes: at least one processor and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the aforementioned data debiasing and alignment enhancement method for a large language model.

[0046] Preferably, a non-transitory computer-readable storage medium stores computer instructions for causing a computer to execute the aforementioned data debiasing and alignment enhancement method for a large language model.

[0047] The present invention discloses the following technical effects:

[0048] This invention provides a data bias removal and alignment enhancement method, electronic device, and storage medium for large language models. By using template matching, sentiment annotation, and conflict group partitioning, it solves the problem of complex and multi-dimensional data bias detection in existing methods, and realizes the transformation of complex bias detection into object and sentiment comparison statistical analysis. By constructing a neutral dialogue model and a bias model, it solves the problems of difficulty in optimizing question-answer pairs during training and the lack of real-time review during use, and realizes automatic generation of erroneous references and optimization of iterative direction during model training. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a schematic diagram of the data debiasing and alignment enhancement process for a large language model provided in an embodiment of the present invention;

[0051] Figure 2 This is a flowchart of data debiasing and alignment enhancement provided in an embodiment of the present invention;

[0052] Figure 3 This is a schematic diagram of the sentiment annotation process provided in an embodiment of the present invention;

[0053] Figure 4 This is a schematic diagram of the biased text and neutral text judgment process provided in an embodiment of the present invention;

[0054] Figure 5 This is a schematic diagram illustrating the training process of the bias model and the neutral dialogue model provided in the embodiments of the present invention. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] The purpose of this invention is to provide a data bias removal and alignment enhancement method, electronic device, and storage medium for large language models, which solves the problems of existing methods such as complex multi-dimensional crossover in data bias detection, difficulty in optimizing question-answer pairs during training, and lack of real-time review during use.

[0057] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] Figure 1 This is a schematic diagram of the data bias removal and alignment enhancement process for a large language model provided in an embodiment of the present invention. Figure 2 The flowchart for data debiasing and alignment enhancement provided in the embodiments of the present invention is as follows: Figure 1 and Figure 2 As shown, this invention provides a data debiasing and alignment enhancement method for large language models, comprising:

[0059] Step 100: Collect text content covering multiple fields to obtain general text;

[0060] Step 200: Set a structured description template, and decompose the general text into single sentences to obtain single-sentence text;

[0061] Step 300: Match the decomposed single-sentence text with the structured description template to obtain the extraction template;

[0062] Step 400: Perform sentiment annotation on the single sentence text according to the extraction template to obtain single sentence sentiment tags; the single sentence sentiment tags include: sentiment text and no sentiment text;

[0063] Step 500: Divide the single-sentence texts in the emotional text into object conflict groups where there is a potential contrast relationship between objects;

[0064] Step 600: Calculate the proportion of each object in each category of sentiment description in each object conflict group to obtain the single object proportion value. Set the single sentence text corresponding to the object conflict group whose single object proportion value does not meet the preset allowable deviation threshold as biased text, and set the single sentence text corresponding to the object conflict group whose single object proportion value meets the preset allowable deviation threshold as neutral text.

[0065] Step 700: Fuse the emotionless text and the neutral text to obtain the pre-biased text, and use the pre-biased text to train the preset large language model to obtain the neutral dialogue model;

[0066] Step 800: Train the preset large language model using the biased text and the emotionless text to obtain the bias model;

[0067] Step 900: Receive user input text, and synchronously input the user input text into the neutral dialogue model and the bias model to obtain the first answer and the second answer;

[0068] Step 1000: Calculate the difference between the first and second answers using a pre-built review formula to obtain the dual-model output difference.

[0069] Step 1100: When the difference between the outputs of the two models is less than the difference standard, return to the step "Receive user input text, and synchronously input the user input text into the neutral dialogue model and the bias model to obtain the first answer and the second answer";

[0070] Step 1200: When the difference between the outputs of the dual models is greater than or equal to the difference standard, the first answer is output to obtain a neutral answer.

[0071] Preferably, a structured descriptive template is set, and the general text is broken down into single sentences to obtain single-sentence text, including:

[0072] The general text is split using the HanLP sentence segmenter based on a multi-dimensional splitting mechanism to obtain the original split text;

[0073] The integrity of the original split text is determined. When the integrity is that the sentence components are incomplete, the current original split text is combined with the adjacent original split texts to obtain the single sentence text.

[0074] Further, the decomposed single-sentence text is matched with the structured description template to obtain the extraction template, including:

[0075] Define the structured description template; the structured description template includes: subject object + sentiment adjective, subject object + behavior description + sentiment adjective, subject object + comparison object + domain type + degree word, subject object + scene limitation + comparison object + degree word;

[0076] The single-sentence text is extracted according to the set element positioning rules to obtain core elements; the core elements include: subject object, comparison object, sentiment adjectives, behavior description, domain type, scene limitation, degree words;

[0077] The structured description template is matched based on the types of elements within the core elements of a single sentence text to obtain the extracted template.

[0078] refer to Figure 3 Based on the extracted template, sentiment annotation is performed on the single-sentence text to obtain single-sentence sentiment tags, including:

[0079] Step 401: Use a part-of-speech tagging tool to perform part-of-speech tagging on the single sentence text, and use SentiWordNet tool to obtain the sentiment score of each word in the single sentence text after part-of-speech tagging, so as to obtain the single sentence sentiment score;

[0080] Step 402: Set the single-sentence text with zero subject objects or emotional adjectives as the emotionless text;

[0081] Step 403: When the number of subject objects or sentiment adjectives is not zero, if the sentiment score of the single sentence belongs to the neutral score range, then the single sentence text is set as emotionless text; otherwise, the single sentence text is set as sentimental text.

[0082] refer to Figure 4The percentage of each object in each category of sentiment description within each object conflict group is calculated to obtain a single object percentage value. The single sentence text corresponding to the object conflict group whose single object percentage value does not meet a preset allowable deviation threshold is set as biased text, and the single sentence text corresponding to the object conflict group whose single object percentage value meets the preset allowable deviation threshold is set as neutral text. This includes:

[0083] Step 601: Extract the maximum value of the single object proportion value in the same object conflict group to obtain the maximum proportion; the expression for the maximum proportion is: ;in, For the first The first group of objects in the conflict group The maximum percentage of each emotional description; This is the function for extracting the maximum value. For the first The first group of objects in the conflict group The object in the first The percentage of a single object in each emotional description;

[0084] Step 602: Calculate the ratio of the percentage of each of the remaining single objects in the same object conflict group to the maximum percentage to obtain the relative percentage; the expression for the relative percentage is:

[0085] ;in, For the first The relative proportion of each object; The object corresponding to the maximum percentage;

[0086] Step 603: Set the single sentence text corresponding to the object conflict group whose relative proportion is greater than or equal to the preset allowable deviation threshold as the neutral text, and set the single sentence text corresponding to the object conflict group whose relative proportion is less than the preset allowable deviation threshold as the biased text.

[0087] refer to Figure 5 The unemotional text and the neutral text are fused to obtain pre-biased text. The pre-biased text is then used to train a pre-defined large language model to obtain a neutral dialogue model, including:

[0088] The pre-biased text is input into an LLMs model to generate question-answer pairs, resulting in the original question-answer pairs.

[0089] The biased answer pairs in the original question-answer pairs are eliminated using a multi-model voting group decision-making method to obtain neutral question-answer pairs; the biased answer pairs have more than 50% of the model voting results indicating bias in the multi-model voting group decision-making process.

[0090] The pre-biased text is masked using autoregressive language modeling and masked language modeling to obtain a pre-training task.

[0091] The pre-set large language model is trained using the neutral question-answer pairs and the pre-training task according to the supervised fine-tuning mechanism, and the parameters of the pre-set large language model are iteratively optimized using the AdamW optimizer to obtain the neutral dialogue model.

[0092] refer to Figure 5 The bias model is obtained by training the pre-defined large language model using the biased text and the emotionless text, including:

[0093] The biased text and the emotionless text are masked using autoregressive language modeling and masked language modeling to obtain a pre-training task.

[0094] The bias model is obtained by training the pre-training task using the unsupervised mechanism to obtain the bias model.

[0095] Furthermore, the pre-built review formula is as follows:

[0096] ;in, Output the difference degree for the two models; , These are the weights for semantic similarity differences and sentiment difference factors, respectively. , These are the semantic embedding vectors of the first answer and the second answer, respectively; , These are the overall sentiment scores for the first and second responses, respectively.

[0097] Specifically, a large amount of general text is collected and broken down into individual sentences; the main subject and the descriptive part describing the main subject are extracted from each sentence; the extracted descriptive parts are sentiment-labeled; based on the labeling, the sentences are divided into two parts: sentiment-labeled text and non-sentiment-labeled text; main subjects with potential opposing or contrasting relationships in the sentiment-labeled text are grouped into the same object conflict group; the proportion of each main subject in different sentiment descriptions within the same object conflict group is counted; if the proportions of different subjects in the same group differ significantly in the same sentiment description, it indicates a "bias" in that sentiment description, and the corresponding text is a non-neutral descriptive text; non-neutral descriptive texts are removed from the original text set to obtain neutral texts.

[0098] Preferably, the multi-dimensional segmentation mechanism includes grammatical structure dimension, semantic coherence dimension, and punctuation standardization dimension. Compared with traditional segmenters, HanLP sentence segmenter, with its adaptability to complex Chinese sentence structures, can identify the core predicate and its governing components through dependency parsing, reducing the segmentation error of ambiguous sentences. The multi-dimensional segmentation mechanism is imported into the HanLP sentence segmenter, and the text is initially segmented according to the priority of grammar, semantics, and punctuation. Using HanLP's part-of-speech tagging tool, the presence of a core subject-predicate structure is detected. A semantic similarity model is used to determine whether a segment requires context for understanding. When the original segmented text has grammatical incompleteness or semantic non-independence, it is combined with neighboring original segmented texts. The combination must follow the principle of minimum scope supplementation: that is, it should be combined with the previous or next text first. If the above problems still exist, the combination domain is expanded sequentially. If combining three texts still does not meet the requirements, the text is judged to have a semantic break, and the corresponding text is removed.

[0099] Furthermore, this embodiment sets up various structured adjective templates based on conventional sentence layouts, including but not limited to: subject object + sentiment adjective, subject object + behavior description + sentiment adjective, subject object + comparison object + domain type + degree word, and subject object + scene limitation + comparison object + degree word. The correspondence between different elements and attribute features is as follows: subject object or comparison object corresponds to noun or noun phrase, sentiment adjective corresponds to adjective, behavior description corresponds to verb or verb phrase, domain type corresponds to noun or noun phrase, scene limitation corresponds to prepositional phrase or noun phrase, and degree word corresponds to adverb. Natural language processing tools such as HanLP and spaCy are used to identify word attributes through part-of-speech tagging, identify logical relationships between words through dependency parsing, and match structured adjective templates according to the types of elements within the core elements of a single sentence text.

[0100] Preferably, SentiWordNet is a sentiment lexicon based on WordNet, retaining only the words corresponding to the core elements in a single sentence and removing function words without sentiment. If there are polysemous words in the core elements, disambiguation is required based on elements such as scene limitation and domain type. SentiWordNet is used to perform equal and weighted calculations on the core sentiment-related words in a single sentence to obtain the sentiment score of the single sentence. If the number of subject objects or the number of sentiment adjectives in the core elements are zero, it is determined that the core elements are missing, and the corresponding text is classified as sentimentless text. Based on the SentiWordNet score distribution and experimental verification, a neutral score interval is set. If the sentiment score of a single sentence falls within the neutral score interval, it is considered sentimentless text; otherwise, it is considered sentimental text.

[0101] Furthermore, the percentage of a single object is calculated using the frequency percentage method. The object described most frequently on a specific emotional dimension within the group is identified, and its percentage is the baseline percentage for that dimension. The extraction formula is as follows:

[0102]

[0103] In the formula For the first The first group of objects in the conflict group The largest proportion of emotional descriptions; This is the function for extracting the maximum value. For the first The first group of objects in the conflict group The object in the first The percentage of a single object in each sentiment description.

[0104] If multiple objects within a group have the same percentage and are all at the maximum value, the largest percentage is taken as the maximum percentage. The difference between the percentage of a non-maximum percentage object and the benchmark object is quantified by relative percentage. A smaller difference indicates a more balanced sentiment description; a larger difference suggests a greater likelihood of bias. The expression for relative percentage is as follows:

[0105]

[0106] In the formula For the first The relative proportion of each object; The object corresponding to the largest proportion.

[0107] Relative proportions can eliminate the influence of the total number and directly reflect the degree of imbalance, making the judgment more objective. If the relative proportion of a single sentence in a conflict group exceeds or equals a preset threshold, it is judged as neutral text; otherwise, it is judged as biased text.

[0108] Optionally, a pre-training task is constructed based on biased and unemotional text using "autoregressive language modeling + masked language modeling". A pre-defined large language model is selected and trained using an unsupervised training mode to obtain a bias model. The fused unemotional and neutral texts are then cleaned, removing fragmented text with a character length of less than 10 characters and abnormal text containing garbled characters or special symbols. The SimHash algorithm is used to deduplicate the text, setting a similarity greater than or equal to 0.95 to determine duplicate text, retaining the first occurrence of the text, resulting in clean pre-biased text. A pre-defined LLMs model is selected, and the clean pre-biased text is input into the model in a paragraph-level input-question-answer generation mode. Each text paragraph generates 2 to 3 sets of question-answer pairs, and all generated results are summarized into an original question-answer pair set. A voting model set containing 3 to 5 pre-trained models with sentiment bias detection capabilities is constructed. The original question-answer pairs are input into each voting model one by one, and the model outputs a judgment result indicating whether a pair is biased or unbiased. When more than 50% of the models determine that a question-answer pair is biased, it is marked as a biased answer pair and removed; the remaining question-answer pairs are considered neutral. A pre-training task is constructed using a combination of autoregressive language modeling and masked language modeling. A pre-set large language model is used as the base model, and a supervised fine-tuning mechanism is used for joint training. The AdamW optimizer is used for iterative parameter optimization, ultimately yielding a neutral dialogue model. During iterative training of the neutral dialogue model, the difference between the output of the bias model and the output of the current iteration step can be simultaneously incorporated into the loss function, further enhancing the neutral narrative capability during training.

[0109] Preferably, a review formula is constructed:

[0110]

[0111] In the formula, Output the difference between the two models; , These are the weights for semantic similarity differences and sentiment difference factors, respectively. , These are the semantic embedding vectors of the first and second answers, respectively. , These are the overall sentiment scores for the first and second responses, respectively.

[0112] The preprocessed user input is simultaneously fed into both the neutral dialogue model and the bias model to obtain the first answer (A) and the second answer (B). The first and second answers are then converted into semantic embedding vectors. These semantic embedding vectors are substituted into the aforementioned review formula to obtain the dual-model output difference score. When the calculated difference score is less than the difference score standard, the system returns to the neutral dialogue model for secondary generation. When the calculated difference score is greater than or equal to the difference score standard, the current first answer is output as the neutral answer.

[0113] As an optional implementation, this embodiment also provides an electronic device, including: at least one processor, and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the aforementioned data debiasing and alignment enhancement method for a large language model.

[0114] As an optional implementation, this embodiment also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the aforementioned data debiasing and alignment enhancement method for large language models.

[0115] The beneficial effects of this invention are as follows:

[0116] This invention transforms complex and cross-cutting bias detection into object and sentiment comparison statistical analysis through template matching, sentiment annotation, and conflict group division, reducing the complexity of bias removal and improving the efficiency of biased text recognition. By constructing a neutral dialogue model and a bias model, it provides erroneous references, avoiding the process of collecting a large number of people's evaluations required for traditional human preference alignment, and optimizing the training of the neutral dialogue model and the review process during operation.

[0117] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0118] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A data debiasing and alignment enhancement method for large language models, characterized in that, The method comprises the following steps: Collecting text content covering multiple fields to obtain general text; Setting a structured descriptive template, and performing single sentence decomposition on the general text to obtain single sentence text; Matching the decomposed single sentence text with the structured descriptive template to obtain an extraction template; Performing sentiment labeling on the single sentence text according to the extraction template to obtain a single sentence sentiment label; The single sentence sentiment label includes emotional text and non-emotional text; Dividing the single sentence text in which the objects in the emotional text have potential contrast relationships into object conflict groups; Statistically analyzing the proportion of each object in each type of emotional description in each object conflict group to obtain a single object proportion value, setting the single sentence text corresponding to the object conflict group whose single object proportion value does not meet a preset allowable deviation threshold as biased text, and setting the single sentence text corresponding to the object conflict group whose single object proportion value meets the preset allowable deviation threshold as neutral text; Fusing the non-emotional text and the neutral text to obtain pre-debiased text, and training a preset large language model using the pre-debiased text to obtain a neutral dialogue model; Training the preset large language model using the biased text and the non-emotional text to obtain a biased model; Receiving user input text, synchronously inputting the user input text into the neutral dialogue model and the biased model to obtain a first answer and a second answer; Calculating the first answer and the second answer using a pre-constructed review formula to obtain a double-model output difference degree; When the double-model output difference degree is less than a difference degree standard, returning to the step of receiving user input text, synchronously inputting the user input text into the neutral dialogue model and the biased model to obtain a first answer and a second answer; When the double-model output difference degree is greater than or equal to the difference degree standard, outputting the first answer to obtain a neutral answer.

2. The data debiasing and alignment enhancement method for large language models according to claim 1, characterized in that, Setting a structured descriptive template, and performing single sentence decomposition on the general text to obtain single sentence text, comprising: Splitting the general text using a HanLP sentence splitter according to a multi-dimensional splitting mechanism to obtain original split text; Judging the content integrity of the original split text, and combining the current original split text with adjacent original split text to obtain the single sentence text when the content integrity is incomplete.

3. The data debiasing and alignment enhancement method for large language models according to claim 1, characterized in that, Matching the decomposed single sentence text with the structured descriptive template to obtain an extraction template, comprising: Setting the structured descriptive template; the structured descriptive template includes: subject object + emotional adjective, subject object + behavior description + emotional adjective, subject object + comparison object + field type + degree word, and subject object + scene limitation + comparison object + degree word; Extracting elements from the single sentence text according to a set element positioning rule to obtain core elements; the core elements include: subject object, comparison object, emotional adjective, behavior description, field type, scene limitation, and degree word; Matching the structured descriptive template according to the types of elements in the core elements in the single sentence text to obtain the extraction template.

4. The data debiasing and alignment enhancement method for large language models according to claim 1, characterized in that, According to the extraction template, the single sentence text is sentiment labeled to obtain a single sentence sentiment label, including: The single sentence text is part-of-speech tagged by using a part-of-speech tagging tool, and a sentiment score of each word in the single sentence text after part-of-speech tagging is obtained by using a SentiWordNet tool to obtain a single sentence sentiment score; The single sentence text with zero number of subject objects or sentiment adjectives is set as the non-emotional text; When the number of subject objects or sentiment adjectives is not zero, if the single sentence sentiment score belongs to a neutral score interval, the single sentence text is set as the non-emotional text, otherwise the single sentence text is set as the emotional text.

5. The data debiasing and alignment enhancement method for large language models according to claim 1, characterized in that, The proportion of each object in each type of sentiment adjective in each object conflict group is counted to obtain a single object proportion value, the single sentence text corresponding to the object conflict group whose single object proportion value does not meet a preset allowable deviation threshold is set as a biased text, and the single sentence text corresponding to the object conflict group whose single object proportion value meets the preset allowable deviation threshold is set as a neutral text, including: extracting a maximum value of the single-object proportion value in the same object conflict group, to obtain a maximum proportion; an expression of the maximum proportion is: ; wherein, is the maximum proportion in the first group of the object conflict group in the first emotional description; is a maximum value extraction function; is the maximum proportion in the first group of the object conflict group in the first object in the first emotional description; The ratio of the remaining single object proportion value to the maximum proportion in the same object conflict group is calculated to obtain a relative proportion; the expression of the relative proportion is: ; wherein, is the relative proportion of the first object; is the relative proportion of the first object; is the object corresponding to the maximum proportion. The single sentence text corresponding to the object conflict group whose relative proportion is greater than or equal to the preset allowable deviation threshold is set as the neutral text, and the single sentence text corresponding to the object conflict group whose relative proportion is less than the preset allowable deviation threshold is set as the biased text.

6. The data debiasing and alignment enhancement method for large language models according to claim 1, characterized in that, The non-emotional text and the neutral text are fused to obtain a pre-debias text, and the pre-debias text is used to train a preset large language model to obtain a neutral dialogue model, including: The pre-debias text is input into an LLMs model to generate a question and answer pair to obtain an original question and answer pair; A biased answer pair in the original question and answer pair is removed by using a multi-model voting group decision method to obtain a neutral question and answer pair; the biased answer pair exceeds 50% of the model voting results as biased when performing multi-model voting group decision; The pre-debias text is masked by using autoregressive language modeling and masked language modeling to obtain a pre-training task; The neutral dialogue model is obtained by training the preset large language model according to a supervised fine-tuning mechanism by using the neutral question and answer pair and the pre-training task, and iteratively optimizing the parameters of the preset large language model by using an AdamW optimizer.

7. The data debiasing and alignment enhancement method for large language models according to claim 1, characterized in that, The biased text and the non-emotional text are used to train the preset large language model to obtain a biased model, including: The biased text and the non-emotional text are masked by using autoregressive language modeling and masked language modeling to obtain a pre-training task; The biased model is obtained by training the preset large language model according to an unsupervised mechanism by using the pre-training task.

8. The data debiasing and alignment enhancement method for large language models according to claim 1, characterized in that, The pre-constructed review formula is: ; wherein, is the difference of the double model outputs; , are semantic similarity difference item weight, sentiment tendency difference item weight, respectively; , are semantic embedding vectors of the first answer and the second answer, respectively; , are overall sentiment scores of the first answer and the second answer, respectively.

9. An electronic device, comprising: including: At least one processor, and a memory connected in communication with the processor; wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor to enable the processor to perform the method for data debiasing and alignment enhancement of large language models according to any one of claims 1-8.

10. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method for data debiasing and alignment enhancement of large language models according to any one of claims 1-8.

Images