Large language model prejudice reduction method and system based on cross-model judgment

Abstract

The invention discloses a large language model prejudice reduction method and system based on cross-model judgment, and the method comprises the steps: giving a group of multiple different LLM models, inputting preset prompts into the different LLM models, and obtaining a response generated by each LLM model; based on the composite prejudice measurement standard, evaluating the prejudice level of the response generated by all other LLM models in the same row group by using each LLM model, and obtaining a prejudice evaluation score; on the basis of the obtained prejudice evaluation scores, calculating a wave arrival score of each response by using an improved wave arrival counting mechanism, and selecting the response with the highest wave arrival score as a consensus target response which is accepted by collectively and has the least prejudice; and performing parameter fine tuning on each participated LLM model based on the obtained low-prejudice consensus target response. The invention provides a novel cross-model judgment framework for detecting and reducing prejudice in LLM by referring to concepts of human companion review and collaborative learning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of large language model bias reduction, and particularly relates to a large language model bias reduction method and system for cross-model adjudication. BACKGROUND

[0002] Modern artificial intelligence systems, particularly LLMs, are increasingly integrated into critical social infrastructure, which requires strict examination of their operational characteristics and potential impacts by the present embodiment. These models now frequently provide information or automate decisions for high-risk decisions that affect millions of people, covering areas from employment screening to financial credit allocation. However, there is growing evidence of a major challenge: current LLMs often inherit, reflect, and possibly amplify human social biases present in their vast training datasets.

[0003] This phenomenon is not just a technical flaw, but also has far-reaching social impacts. For example, commercial and open-source models exhibit statistically significant biases, such as consistently favoring female characters in short stories or associating African American Vernacular English with negative or criminal contexts at significantly higher frequencies than Standard American English dialects. Such biases can cause real harm in various fields. In employment, biased LLMs used for resume screening or job recommendation systems can lead to discriminatory hiring practices based on gender, race, or age, consistently discriminating against certain demographic groups. In the financial sector, models used for loan approval or credit scoring can lead to unfair outcomes and limit access to capital for minority groups. In the healthcare field, biased LLMs providing information for medical diagnosis or treatment recommendations can exacerbate existing health disparities related to gender or race. Furthermore, in the criminal justice system, risk assessment tools using LLMs for parole or sentencing decisions can reinforce systemic biases, leading to unfair outcomes for certain groups.

[0004] Although this problem has been widely recognized, current bias detection and reduction methods face substantial limitations, often providing only a partial view of the complex nature of LLM bias. Supervised fine-tuning, as a commonly used method, relies heavily on human annotation, which not only has high acquisition costs but also may inadvertently embed the annotators' implicit or explicit biases into the model. Adversarial filtering techniques aim to remove sensitive attribute information, but may compromise model utility by stripping valuable context or cultural nuances. In addition, post-processing correction methods applied after model training often introduce significant computational overhead, increasing inference latency, and mainly address symptoms rather than the root of biases embedded in the core representation of the model.

[0005] At its core is a key weakness that pervades the current evaluation paradigm: the circular validation problem. In many cases, models are evaluated using metrics, benchmarks, or even simulated interactions that are similar to those used in their own training or development processes. This self-referential evaluation loop creates a serious blind spot, particularly for biases that stem from shared architectural choices, pre-training data, or fine-tuning objectives. For example, external audits have shown that well-known models like LLaMA2 perform significantly worse compared to the results reported by their internal self-evaluation procedures.

[0006] The invention directly addresses a key question: Can LLMs collectively identify and correct biases that no single dataset or model can perceive on its own? The hypothesis is that, just as democratic deliberation can reveal truths that no single individual can grasp, structured collaboration among heterogeneous models can expose implicit biases through adversarial review. The key insight lies in architectural pluralism—by coordinating different models to critique each other’s outputs, the invention creates a correction mechanism where biases cancel each other out while true consensus emerges. SUMMARY

[0007] The invention provides a cross-model adjudication large language model bias reduction method and system to solve the problem that current bias detection and reduction methods usually fine-tune LLMs based on a large number of datasets and metrics, which due to inherent inflexibility (fixed test content, such as text) and potential circumvention (benchmarks themselves may have been used to train models), cannot achieve effective bias elimination.

[0008] According to a first aspect, in an embodiment, a cross-model adjudication large language model bias reduction method is provided, the method comprising: Given a set of multiple different LLM models, inputting a preset prompt into each of the different LLM models to obtain a response generated by each LLM model; Based on a composite bias metric, using each LLM model to evaluate the bias level of the responses generated by all other LLM models in the peer group, and obtaining a bias evaluation score; Based on the obtained bias evaluation score, using an improved Borda count mechanism to calculate the Borda score of each response, and selecting the response with the highest Borda score as the consensus target response with the least bias recognized by the group; Based on the obtained consensus target response with low bias, performing parameter fine-tuning on each participating LLM model.

[0009] Further, given a set of multiple different LLM models, inputting a preset prompt into each of the different LLM models to obtain a response generated by each LLM model, specifically comprising: A set of N different LLM models is represented as M = {M1, M2, …, MN}, where N is the number of models in the set. The preset prompt is input into each of the different LLM models in the set M to obtain a response generated by each LLM model in the set M.N}; For a given input prompt q, each model M m generates a response r m = (y1, y2,..., y L L), L is the sequence length.

[0010] Further, given a set of multiple different LLM models, the preset prompt is input into different LLM models respectively to obtain responses generated by each LLM model, specifically including: The generated responses use random sampling to encourage diversity while maintaining coherence, specifically using kernel sampling combined with temperature scaling: ; where z m,t,yt is the logits produced by model M m at time step t for token y t , τ is a temperature parameter.

[0011] Further, based on a composite bias metric, each LLM model is used to evaluate the bias level of the responses generated by all other LLM models in the peer group, specifically including: The composite bias metric provides a set of guiding principles or conceptual dimensions presented through the prompt prompt for the LLM model; model M j evaluates the bias level of the original response r k according to different dimensions of the established composite bias metric, including relevance, vocabulary, and context, k ∈ {1,..., N}, j ≠ k, N is the number of LLM models.

[0012] Further, based on a composite bias metric, each LLM model is used to evaluate the bias level of the responses generated by all other LLM models in the peer group, specifically including: The relevance bias dimension is used to prompt the model M j to evaluate whether the response r k implicitly or explicitly establishes a stereotype association between a demographic group and a specific attribute, role, or concept reflecting social bias, and the corresponding dimension aims to capture the characteristics of association bias measured in the embedding space, by requiring the LLM to identify subtle semantic links; The vocabulary bias dimension is used to prompt the model M j to identify whether there is biased, stereotypical, derogatory, or harmful language in the response r k , including checking words or phrases associated with negative stereotypes or unfair generalizations, and using the language knowledge of the LLM to detect corresponding class problem terms in context; Contextual bias dimensions are used to prompt the guiding model M j The overall narrative, sentiment, and implied meaning of the responses r k , including assessing whether the text subtly reinforces stereotypes, presents different groups in an unbalanced or unfair way, or normalizes biased views even if no biased terms are explicitly used; the corresponding dimensions aim to capture features that deviate from neutrality or fair distribution in a broader context.

[0013] Further, in evaluating the bias level of all other LLM models' generated responses in the peer group using each LLM model based on the composite bias metric, the method specifically further comprises: Guided revision: prompting the model M j to generate a revised version of the responses r k , denoted as , aiming to minimize the biases identified in the evaluation step while preserving the original intent and core information; Counterfactual generation: prompting the model M j to generate alternative versions of the modified versions by systematically changing sensitive attributes .

[0014] Further, obtaining the bias evaluation score s specifically comprises: jk The comprehensive analysis of the model M j : ; wherein, represents the process of the model M j evaluating the responses r , represents the numerical scores output by the model when evaluating the responses r according to different dimensions of the composite bias metric, is an optional confidence weight of the model , represents a small amount of Gaussian noise added to the score, and σ is a hyperparameter; The output of this stage is an N x N score matrix S, which reflects the collective judgment of the bias level of the original responses {r1,..., r N}, and the generated revisions and counterfactuals are collected as supplementary data.

[0015] Furthermore, based on the bias assessment score, the improved bod score counting mechanism is used to calculate the bod score of each response, and the response with the highest bod score is selected as the consensus target response with the least bias recognized by the collective, specifically including: For each response r k , its arrival fraction b k It is calculated based on the pairwise comparison of each evaluator model ranking, specifically, for each evaluator model M j , based on the bias evaluation score s ji For all responses {r i} i≠j Ranking, response r k From the estimator model M j The points earned are based on M j Bias assessment score s ji , how many other responses it beats i Depends on: ; where I(·) is the indicator function; b k The calculation formula summarizes the response r k In all evaluator models M except self-evaluation j The number of wins in pairwise comparisons; With the highest arrival score b k Response Selected as consensus target response: ; as a low-bias target response distilled from the collective judgment of the model population for a given cue q.

[0016] Furthermore, based on the obtained low-bias consensus target response, the parameters of each participating LLM model are fine-tuned, including: Adopt LoRA mechanism to efficiently fine-tune model parameters: For each model M m , LoRA introduces a low-rank decomposition matrix B for the specific weight matrix W0 in the original model m and A m ; Model updates are limited to ; Model M m The fine-tuning goal is to minimize a distillation loss, that is, the difference between the model's predicted distribution of response tokens and the target consensus response r ∗ The cross entropy loss between: ; in It's a reminder and its corresponding consensus target response is the standard cross-entropy loss of sequence generation, and the original model weights W0 remain frozen. is the standard cross-entropy loss of sequence generation, and the original model weights W0 remain frozen.

[0017] According to a second aspect, in an embodiment, a cross-model adjudication large language model bias reduction system is provided, the system comprising: a response generation module configured to, for a given set of multiple different LLM models, input a preset prompt to each of the different LLM models to obtain a response generated by each of the LLM models; a cross-model evaluation module configured to, based on a composite bias metric, evaluate a bias level of the responses generated by all other LLM models in the peer group using each of the LLM models, and obtain a bias evaluation score; a consensus extraction module configured to, based on the obtained bias evaluation score, calculate a Borda score of each response using an improved Borda count mechanism, and select a response with the highest Borda score as a consensus target response with the least bias that is collectively recognized; a model fine-tuning module configured to, based on the obtained consensus target response with the least bias, fine-tune parameters of each of the participating LLM models.

[0018] The present application provides a cross-model adjudication large language model bias reduction method and system, which has the following beneficial effects: 1) A cross-model peer review protocol is proposed, in which multiple LLMs iteratively evaluate each other's responses using dynamically calibrated metrics; 2) A consensus extraction algorithm is proposed to identify the most unbiased output through an improved Borda count; 3) Parameter efficient adaptation, fine-tuning each model with these extracted unbiased consensus, only adding 0.3% additional parameters.

[0019] The cross-model adjudication framework of the present application does not require human annotation, preserves cultural specificity, and operates as a general bias "perspective" applicable to any LLM architecture. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A flowchart of a cross-model adjudication large language model bias reduction method according to an embodiment of the present application is provided; Figure 2 A logic structure schematic diagram of a cross-model adjudication large language model bias reduction system according to an embodiment of the present application is provided. DETAILED DESCRIPTION

[0021] The present invention will be further described in detail below by means of specific embodiments in conjunction with the accompanying drawings. Similar elements in different embodiments are numbered with associated similar elements. In the following embodiments, many detailed descriptions are provided to enable the present invention to be better understood. However, those skilled in the art will readily appreciate that some of the features may be omitted under different circumstances, or may be replaced by other elements, materials, or methods. In some cases, some operations related to the present invention are not shown or described in the specification. This is to avoid overwhelm the core of the present invention with excessive descriptions. However, for those skilled in the art, it is not necessary to describe these related operations in detail. They will fully understand the related operations based on the description in the specification and the general technical knowledge in the art.

[0022] In addition, the features, operations, or characteristics described in the specification may be combined in any appropriate manner to form various embodiments. Furthermore, the steps or actions in the method description may be reordered or adjusted in a manner readily apparent to those skilled in the art. Therefore, the various sequences in the specification and drawings are provided solely for the purpose of clearly describing a particular embodiment and are not intended to be mandatory, unless otherwise specified.

[0023] The first embodiment of the present invention provides a large language model bias reduction method for cross-model arbitration, which uses Lissajous trajectories to visualize current signals, captures the characteristics of series arc faults under different load conditions, determines the power mode type based on the observed current harmonic components, relies on abnormal current pulses to start, and finally completes series arc fault detection based on image recognition technology. Figure 1 Provide detailed explanation.

[0024] like Figure 1 As shown, in step S100, given a set of multiple different LLM models, preset prompts are input into different LLM models respectively to obtain responses generated by each LLM model.

[0025] Response generation: Each model M m ∈M independently generates a response r for an input prompt q m .

[0026] The above steps specifically include: S110, for a given input prompt q, each model M m Generate a response r m =(y1,y2,...,y L ), where L is the sequence length.

[0027] S120, random sampling is used to encourage diversity while maintaining consistency. Specifically, this embodiment uses kernel sampling combined with temperature scaling: ; where z m,t,yt It is model M m At time step t, the word y t The logits generated are given by [ tau ], where τ is the temperature parameter. Kernel sampling is applied with a cumulative probability threshold p = 0.9, retaining the smallest set of tokens whose cumulative probability exceeds p. This ensures a diverse but plausible continuation of the correlation with the prompt q.

[0028] like Figure 1 As shown, in step S200, based on the composite bias metric, each LLM model is used to evaluate the bias level of responses generated by all other LLM models in the peer group, and a bias evaluation score is obtained.

[0029] Cross-model evaluation: Each model M j Evaluate all other models in the peer group {r k} k≠j This evaluation uses the Composite Bias Metric (CBM) proposed in this embodiment to evaluate each response r k Crucially, M j Do not evaluate your own response r j , to prevent self-assessment bias.

[0030] The above steps specifically include: This phase implements the core peer review mechanism, which includes not only evaluation but also guided revision and counterfactual generation. k (k∈{1,...,N}), each other model M j (j=k) executes a multi-step task prompted by a specific instruction: S210, Bias Assessment: Model M j First, the original response r is evaluated according to the dimensions of the composite bias measure (CBM) (relevance, vocabulary, context). k level of bias.

[0031] Composite Bias Metric (CBM): The core of cross-model evaluation is to guide each evaluator model M j Evaluate peer responses k To provide a structured and multidimensional basis for this assessment, this embodiment defines CBM as a set of guiding principles or dimensions presented to the evaluator LLM through its prompts, rather than an externally calculated score. Specifically, M jOverall assessment of r k according to the three key potential bias dimensions (association, lexical, contextual), as follows: 1) Association bias dimension (semantic stereotypes): used to prompt the instructing model M j to assess whether the response r k implicitly or explicitly establishes stereotype associations between demographic groups (e.g., gender, race, age) and specific attributes, roles, or concepts that reflect social biases. This dimension aims to capture the characteristics of association bias that are typically measured in embedding spaces, by requiring the LLM to identify subtle semantic links.

[0032] 2) Lexical bias dimension (explicit language): used to prompt the instructing model M j to identify whether there is overtly biased, stereotypical, derogatory, or harmful language in the response r k . This includes checking for words or phrases associated with negative stereotypes or unfair generalizations, and leveraging the LLM's language knowledge to detect such problematic terms in context.

[0033] 3) Contextual bias dimension (implicit bias and sentiment): used to prompt the instructing model M j to assess the overall narrative, sentiment, and implied meaning of the response r k . This includes assessing whether the text subtly reinforces stereotypes, presents different groups in an unbalanced or unfair manner, or normalizes biased viewpoints even without explicit use of biased language. This dimension aims to capture characteristics that measure deviation from neutrality or fair distribution in a broader context.

[0034] The assessment prompts provided to M j can take various forms, such as: • After presenting the CBM dimensions as criteria, ask for a single, overall bias score (e.g., on a scale of 1 to 10, with 1 indicating the least bias).

[0035] • Ask for separate scores for each dimension, then aggregate (e.g., using a weighted average of λ E , λ L , λ C ) by M j itself in subsequent steps or externally to calculate the final score s jk .

[0036] In the implementation of this embodiment, the overall scoring approach is primarily used, instructing the assessor M j to provide a single score representing the overall bias level of r k , derived from its assessment of these CBM dimensions.

[0037] Therefore, the bias assessment score calculated later Representative model M j After being prompted to assess r according to CBM criteria k The numerical score output when is . It is an LLM-generated assessment designed to capture the multifaceted nature of bias, rather than a direct computation of a mathematical formula. This approach fully leverages the contextual understanding and reasoning capabilities of the LLM for peer review and directly aligns with the core concept of cross-model adjudication.

[0038] S220, Modification / Revision: Tips to clarify the requirement of M j Generate r k A revised version of , aims to minimize the bias identified in the evaluation step while preserving the original intent and core message. For example, the instruction: "Please rewrite the following text to remove any detected bias while remaining faithful to the original theme: 'r k '. " S230, Counterfactual Generation: Prompts may also require M j Generate (ideally revised) text by systematically changing sensitive attributes (e.g., replacing "man" with "woman," "doctor" with "nurse" for gender analysis, or changing names / pronouns associated with different races or ages) This helps detect fairness and consistency across demographic groups. Denoting these as For example, the instruction: "Now, generate revised versions of the text corresponding to gender = 'female', age = 'elderly', race = 'Asian', ...." Therefore, the evaluation score s jk From M j Comprehensive analysis: ; in Represents model M j Evaluate the response r k In practice, this involves prompting M j Scoring r based on CBM dimension instructions k . Denotes the evaluator M j Calculated or estimated response r k This implicitly involves M j Evaluate r on the embedding, vocabulary, and context dimensions of CBM k .w j is the evaluator M j Optional confidence weights for . Initially uniform (w j =1 / N), we can calculate the jThe historical consistency with the consensus is updated dynamically, although static weights are used for simplicity in the main experiments. represents a small amount of Gaussian noise added to the scores. This acts as a random regularization technique that may improve robustness and prevent degenerate evaluation strategies where the model may collude. σ is a small hyperparameter. The output of this stage is still the N×N score matrix S (ignoring s jj ), which reflects the original response {r1,...,r N}Collective judgment of bias level. Generated revisions and counterfactuals Collected as supplementary data.

[0039] like Figure 1 As shown, in step S300, based on the bias assessment score obtained, the improved arrival counting mechanism is used to calculate the arrival score of each response, and the response with the highest arrival score is selected as the consensus target response with the least bias recognized by the collective.

[0040] Consensus extraction: The evaluation scores obtained in step S200 are aggregated using an improved bod count mechanism to identify the consensus target response r ∗ The response r ∗ represents the “least biased” output of the collective agreement with the prompt q and serves as the target for subsequent fine-tuning.

[0041] The above steps specifically include: S310, given a peer evaluation matrix S, the goal is to identify the response rk that is collectively judged to be the least biased. This embodiment uses Borda counting, a voting method known for its robustness to strategic manipulation and its tendency to select consensus candidates. For each response r k , its arrival fraction b k is calculated based on the pairwise comparisons in each evaluator ranking. Specifically, for each evaluator M j , according to the score s ji For all responses {r i} i≠j Rank. Response r k From the evaluator M j The points earned are based on M j Ratings ji , how many other responses it beats i (i≠j,k). Assuming that the lower the bias assessment score, the better (less bias): ; where I(·) is the indicator function. This formula summarizes the response r k Number of “wins” in pairwise comparisons of all evaluators j (excluding self-evaluation j=k).

[0042] S320, with the highest arrival score b k The response r ∗ Selected as consensus target: ; This r ∗ As a low-bias target example extracted from the collective judgment of the model population for a given prompt q. It is used as the ground truth for fine-tuning in the next stage.

[0043] Consensus mechanism analysis: The effectiveness of this embodiment of the cross-model arbitration framework depends on whether the response chosen by the consensus mechanism is less biased than a typical single output. This section provides a theoretical basis for achieving this improvement through consensus.

[0044] make CBM (r m ) is the model M m Response m The true bias score of . Let s jk M j Give r k Assume that the evaluation score is a noisy estimate of the true bias, that is, E[ ]= CBM (r k )(in The Borda Count is considered to be Condorcet efficient under standard voting theory assumptions, meaning that if the response r k If it is able to beat all other responses in pairwise comparisons with most evaluators, it will likely be selected by the Borda Count.

[0045] Theorem 1 (Consensus Improvement Bound): Assume that the evaluation score s jk Bias against truth CBM (r k ) provides an unbiased estimate with bounded variance and reasonable independence of the estimator errors. Then, the consensus response r selected by the Borda count is ∗ The expected bias of ; where δ(N) is an error term that decreases as the number of diverse evaluators N increases. With high probability, the selected response r ∗ The bias score will be less than or equal to all generated responses {r m}Minimum bias score in .

[0046] Proof: This result relies on the law of large numbers applied to the aggregated scores. As more diverse evaluators contribute, independent noise or bias in individual evaluations tends to average out, making the Borda ranking tend towards the true underlying biased ranking. The Borda count aggregates pairwise comparisons, and under the independence assumption, the probability of choosing a suboptimal candidate decreases rapidly with N. The pairwise win probability in the Borda score computation can be formally bounded using concentration inequalities such as the Hoeffding inequality.

[0047] Lemma 1 (Integration Advantage): If models M m exhibit sufficiently diverse biases (e.g., stemming from different datasets or architectures), consensus mechanisms are more likely to identify and filter out idiosyncratic biases that exist in only a subset of models Lemma 1 (Integration Advantage). If models M m exhibit sufficiently diverse biases (e.g., stemming from different datasets or architectures), consensus mechanisms are more likely to identify and filter out idiosyncratic biases that exist in only a subset of models. The collective judgment benefits from this diversity, leading to consensus errors that generally perform better than relying on a single best model, possibly decaying at a rate of under simplifying assumptions.

[0048] As shown in FIG. 4, in step S400, based on the obtained low-bias consensus target response, parameter fine-tuning is performed on each participating LLM model. Figure 1

[0049] The above steps specifically include: The present embodiment performs parameter efficient fine-tuning through Low-Rank Adaptation (LoRA), specifically as follows: The final step of cross-model adjudication is to use the low-bias consensus-derived response r ∗ to fine-tune each participating model M m . To maintain computational feasibility and preserve the generalizability of the model, the present embodiment employs LoRA, a parameter-efficient fine-tuning technique.

[0050] For each model M m , LoRA introduces a low-rank decomposition matrix B d×k ∈ R m and A d×r ∈ R m for a particular weight matrix W0∈ R r×k (typically an attention layer). The model update is restricted to where the rank . Only A m and B m are trained, significantly reducing the number of trainable parameters.

[0051] Model M m ​The fine-tuning objective is to minimize a distillation loss, typically the cross-entropy loss between the model’s predicted distribution over response tokens and the target consensus response r ∗ : ; where is the prompt and its corresponding consensus response pair generated by cross-model arbitration, is the standard cross-entropy loss for sequence generation. The original model weights W0 are kept frozen. The specific algorithmic process is as follows:

[0052] Lemma 1 (Parameter Efficiency): For a weight matrix W0∈R d×k , full fine-tuning requires updating dk parameters. LoRA only needs to update r(d+k) parameters in A m and B m . Typically , thus saving parameters significantly (e.g., 0.1%-1% of total parameters). This makes the fine-tuning stage computationally efficient.

[0053] Experimental Example: This section details the experimental setup, evaluation metrics, and comprehensive results demonstrating the effectiveness of the cross-model arbitration framework (cross-model arbitration) in detecting and mitigating large language model bias. This example evaluates the performance of cross-model arbitration relative to baseline models and conducts an ablation study to understand the contributions of its key components.

[0054] Table 1 Cross-model arbitration computational complexity analysis (per prompt)

[0055] where N: number of models. L, L’: sequence length for generation / evaluation. C infer : cost per inference. Parallel complexity assumes sufficient resources.

[0056] Experimental Setup: 1) Hardware and Software Environment All experiments were conducted on a cluster of machines running Ubuntu 20.04.6 LTS. The software stack includes Python 3.9.12, PyTorch 1.11.0+cuda11.3, and Transformers library version 4.50.0. To perform parameter-efficient fine-tuning, this example uses Unsloth version 2025.3.19, which optimizes the LoRA implementation 2) Model Specifications This example selected four state-of-the-art LLMs with 7-9B parameters as experimental subjects, which represent different architectural foundations and pre-training objectives. This diversity is crucial for the cross-model adjudication mechanism of cross-model adjudication, as described in Section 1. The detailed information of the models is shown in Table 2. All models are loaded with their original, publicly released fine-tuned versions. To improve memory efficiency in the multi-model evaluation phase, this example uses 8-bit quantization when loading the model, and utilizes gradient checkpointing to manage GPU memory usage during fine-tuning.

[0057] Table 2. Model specifications, training details, and optimization techniques

[0058] 3) Bias evaluation and prompt generation dataset This example uses HolisticBias, a large dataset designed to evaluate the social bias of language models in 13 protected population categories, including gender, religion, nationality, and age. It contains about 600 identity token embeddings inserted into template sentence structures, resulting in about 450,000 bias probe examples. This example uses HolisticBias to both construct initial prompts for response generation for cross-model adjudication (especially by focusing on incomplete sentence templates for religion and nationality) and to evaluate residual bias after mitigation. In addition, this example refers to the expert-labeled severity scores provided in HolisticBias to verify the bias mitigation indicators of this example.

[0059] 4) Evaluation metrics Given a template prompt T[·] instantiated with an identity token g1from group G k , this example defines the bias of a model as a function of the likelihood function assigned to these sentences. Let L lik (g i ) = -log P(g i | T) represent the negative log-likelihood assigned by the model when completing the template containing the identity g1(i.e., the token-level surprise or pseudo-perplexity). To evaluate whether the model exhibits biased behavior between different groups (G i , G j ), this example performs a non-parametric Mann-Whitney U test on the token-level likelihood distributions L CBM (g) associated with each group: U i,j = MannWhitney(L lik (G i ), L lik (G j)). This example reports whether the differences in likelihood distributions between identity groups are statistically significant (typically p < 0.05), which indicates that the model has a bias in ranking or interpreting prompts that involve different demographic terms.

[0060] Implementation details: 1) Response generation phase For phase 1 (response generation), each model generates completions for incomplete sentences with a maximum sequence length of 128 wordpieces. Kernel sampling (p = 0.9) and temperature τ = 0.7 are used for random sampling to encourage diverse and coherent outputs.

[0061] 2) Cross-model evaluation phase In phase 2, each evaluator model M j evaluates the responses r k generated by its peers (where j = k). The evaluation prompt explicitly instructs M j to consider the CBM dimensions and provide a single overall bias score from 1 to 5, where 1 indicates the most severe bias and 5 indicates the least severe bias.

[0062] 3) Consensus distillation and fine-tuning data preparation Phase 3 involves aggregating the peer evaluation scores to obtain a consensus-derived "unbiased" output for fine-tuning. For each prompt, the responses of all models are scored by their peers. Then, this example applies a modified Borda count mechanism to determine the response with the least overall bias for that particular prompt. In cases where multiple models obtain the same highest Borda score, a response is randomly selected from among them as the fine-tuning target. If an evaluator model fails to provide a score (e.g., due to reasoning errors), its score for that particular response is conservatively set to 3 (neutral) to minimize interference with the consensus mechanism.

[0063] 4) Parameter-efficient fine-tuning This example uses LoRA to perform parameter-efficient fine-tuning of each model M m . The key hyperparameters for LoRA fine-tuning are as follows: • LoRA rank (r): 16 • LoRA alpha (a): 16 (scaling factor for LoRA weights) • LoRA dropout: 0 (no dropout applied to LoRA weights) • Learning rate: 2e-4 • Per-device training batch size: 2 Gradient accumulation steps: 4 (effective batch size of 8) • Maximum sequence length: 2048 (applies to input prompts and target responses) • Number of epochs: 8 • Quantization: 'load_in_4bit=True' for memory-efficient loading of base model.

[0064] These settings are implemented using the unsloth library to optimize performance.

[0065] The method of detecting and mitigating bias in LLMs through collaborative evaluation and refinement proposed by the embodiments of the present application. The cross-model adjudication framework conducts peer review among different LLM models to identify and correct biases that individual models or static benchmarks may not be able to detect. The framework includes three consecutive stages: response generation, cross-model evaluation, and consensus refinement, followed by parameter-efficient fine-tuning using the refined consensus output.

[0066] This process ensures: • Mutual review: biases that one model may overlook can be identified by other models with different training data or architectures.

[0067] • Asymmetric evaluation: models cannot access their own internal states when evaluating peers in the evaluation step, promoting objective evaluation.

[0068] • Information isolation: models do not share any internal model parameters or gradients between them during the evaluation phase, maintaining model independence.

[0069] • Adaptive consensus: the framework identifies the response with the least bias based on collective judgment, rather than relying on a fixed definition of "unbiased."

[0070] Conceptually, the cross-model adjudication framework transforms bias mitigation into a collaborative refinement process, with models leveraging their diverse perspectives to converge on less biased representations and outputs.

[0071] Corresponding to the above disclosed bias mitigation method for large language model of cross-model adjudication, the embodiments of the present application also disclose a bias mitigation system for large language model of cross-model adjudication, as shown in Figure 2 which specifically comprises: a response generation module, configured to input a preset prompt into different LLM models to obtain responses generated by each LLM model, given a plurality of different LLM models; a cross-model evaluation module, configured to evaluate the bias level of responses generated by all other LLM models in the peer group using each LLM model based on a composite bias metric, and obtain bias evaluation scores; a consensus refinement module, configured to calculate the Borda score of each response based on the obtained bias evaluation scores using an improved Borda count mechanism, and select the response with the highest Borda score as the consensus target response with the least bias recognized by the collective; A model fine-tuning module is configured to fine-tune each participating LLM model based on the obtained low-bias consensus target response.

[0072] It should be noted that the detailed description of the cross-model adjudication large language model bias reduction system provided by the embodiments of the present application can refer to the related description of the cross-model adjudication large language model bias reduction method provided by the embodiments of the present application, which will not be repeated here.

[0073] The above application of specific examples to the present invention is used to help understand the present invention and does not limit the present invention. For those skilled in the art to which the present invention belongs, according to the idea of the present invention, a number of simple deductions, deformations or substitutions can be made.

Claims

1. A large language model bias reduction method for cross-model arbitration, characterized by: The method comprises: Given a set of multiple different LLM models, input preset prompts into different LLM models respectively to obtain the responses generated by each LLM model; Based on the composite bias metric, each LLM model is used to evaluate the bias level of responses generated by all other LLM models in the peer group and obtain a bias assessment score; Based on the bias assessment score, the improved boda counting mechanism is used to calculate the boda score of each response, and the response with the highest boda score is selected as the consensus target response with the least bias recognized by the collective; Based on the obtained low-bias consensus target response, parameters of each participating LLM model are fine-tuned.

2. The method for reducing bias in a large language model for cross-model arbitration according to claim 1, wherein: Given a set of multiple different LLM models, input the preset prompts to different LLM models and obtain the responses generated by each LLM model, including: A set of N different LLM models is represented as M={M1,M2,...,M N }; For a given input prompt q, each model M m ∈M generates a response r m =(y1,y2,...,y L ), L is the sequence length.

3. The method for reducing bias in a large language model for cross-model arbitration according to claim 2, wherein: Given a set of multiple different LLM models, input the preset prompts to different LLM models and obtain the responses generated by each LLM model, including: The generated responses are randomly sampled to encourage diversity while maintaining coherence, using kernel sampling combined with temperature scaling: ； where z m,t,yt It is model M m At time step t, the word y t The generated logits, τ is the temperature parameter.

4. The method for reducing bias in a large language model for cross-model arbitration according to claim 1, wherein: Each LLM model is evaluated for the level of bias in responses generated by all other LLM models in the peer group based on a composite bias metric, including: The composite bias metric provides a set of guiding principles or conceptual dimensions for the LLM model through prompts; Model M j The original response is evaluated according to different dimensions of the established composite bias metric, including relevance, vocabulary, and context. k The bias level of , k∈{1,...,N}, j ≠ k, N is the number of LLM models.

5. The method for reducing bias in a large language model for cross-model arbitration according to claim 4, wherein: Each LLM model is evaluated for the level of bias in responses generated by all other LLM models in the peer group based on a composite bias metric, including: The relevance bias dimension is used to prompt the indicator model M j Evaluate the response r k Whether stereotypical associations are implicitly or explicitly established between demographic groups and specific attributes, roles, or concepts that reflect social biases, the corresponding dimension aims to capture the characteristics of association biases measured in the embedding space by requiring the LLM to identify subtle semantic links; Lexical bias dimensions are used to prompt the indicator model M j Identification response r k Whether there is overt bias, stereotyping, derogatory, or harmful language in the content, including examining words or phrases associated with negative stereotypes or unfair generalizations, and using the LLM's linguistic knowledge to detect problematic terms in context; Contextual bias dimensions are used to prompt guidance model M j Evaluate the response r k The dimensions assess the overall narrative, sentiment, and implicit meaning of a text, including whether it subtly reinforces stereotypes, presents different groups in an unbalanced or unfair way, or normalizes biased views, even without explicitly using biased terms; these dimensions are designed to capture features that measure deviations from a neutral or fair distribution in a broader context.

6. The method for reducing bias in a large language model for cross-model arbitration according to claim 4, wherein: When evaluating, using each LLM model, the bias level of responses generated by all other LLM models in the peer group based on the composite bias metric, the method further comprises: Guidance revision: Tips clarify requirements for Model M j Generate response r k A revised version of , which aims to minimize the biases identified in the evaluation step while preserving the original intent and core message; Counterfactual Generation: Prompt Requirement Model M j Generate modified versions by systematically changing sensitive properties Alternative version of .

7. The method for reducing bias in a large language model for cross-model arbitration according to claim 6, wherein: Receive a bias assessment score, including: Evaluation scores jk From M j Comprehensive analysis: ； in, Representation Model Evaluate Response process, Representation Model Evaluating responses on different dimensions based on a composite bias metric The numerical score output when It's a model Optional confidence weights for , represents a small amount of Gaussian noise added to the score, is a hyperparameter; The output of this stage is an N×N score matrix S, which reflects the scores of the original responses {r1,...,r N }Collective judgment of bias level, resulting in revisions and counterfactuals Collected as supplementary data.

8. The method for reducing bias in a large language model for cross-model arbitration according to claim 7, wherein: Based on the bias assessment score, the improved bod score of each response is calculated using the bod score counting mechanism, and the response with the highest bod score is selected as the consensus target response with the least bias recognized by the collective. Specifically, For each response r k , its arrival fraction b k It is calculated based on the pairwise comparison of each evaluator model ranking, specifically, for each evaluator model M j , based on the bias evaluation score s ji For all responses {r i } i≠j Ranking, response r k From the estimator model M j The points earned are based on M j Bias assessment score s ji , how many other responses it beats i Depends on: ； where I(·) is the indicator function; b k The calculation formula summarizes the response r k In all evaluator models M except self-evaluation j The number of wins in pairwise comparisons; With the highest arrival score b k Response Selected as consensus target response: ； as a low-bias target response distilled from the collective judgment of the model population for a given cue q.

9. The method for reducing bias in a large language model for cross-model arbitration according to claim 1, wherein: Based on the obtained low-bias consensus target response, the parameters of each participating LLM model are fine-tuned, including: Adopt LoRA mechanism to efficiently fine-tune model parameters: For each model M m , LoRA introduces a low-rank decomposition matrix B for the specific weight matrix W0 in the original model m and A m ; Model updates are limited to ; Model M m The fine-tuning goal is to minimize a distillation loss, that is, the difference between the model's predicted distribution of response tokens and the target consensus response r ∗ The cross entropy loss between: ； in It's a reminder and its corresponding consensus target response right, is the standard cross entropy loss for sequence generation, and the original model weights W0 remain frozen.

10. A large language model bias reduction system for cross-model arbitration, characterized in that The system comprises: A response generation module is used to input preset prompts into different LLM models respectively given a set of multiple different LLM models to obtain responses generated by each LLM model; A cross-model evaluation module is used to evaluate the bias level of responses generated by each LLM model against all other LLM models in the peer group based on a composite bias metric and obtain a bias evaluation score; A consensus extraction module is used to calculate the boda score of each response based on the bias assessment score obtained by using an improved boda counting mechanism, and select the response with the highest boda score as the consensus target response with the least bias recognized by the collective; The model fine-tuning module is used to fine-tune the parameters of each participating LLM model based on the obtained low-bias consensus target response.

Images