Medical large language model multi-dimensional value alignment method and system based on low-rank multi-subspace

Abstract

The application discloses a medical large language model multi-dimensional value alignment method and system based on low-rank multi-subspace, first constructs a plurality of medical value dimension definition preference corpus sets, and constructs a plurality of mutually orthogonal low-rank medical value subspaces corresponding to each dimension based on the corpus sets; then extracts the intermediate layer hidden vector of the clinical input to project to each sub-space to obtain the current value state, and constructs the optimization direction based on the preference and the non-preference answer; subsequently, a low-rank parameter update module subject to geometric constraints is constructed for each sub-space at the large language model target parameter layer, and the outputs of each module are dynamically weighted according to the input situation through a cross-subspace coordination gating network; finally, an end-to-end fine-tuning is performed by using a joint loss function. Through the orthogonal sub-space decoupling and the dynamic gating mechanism, the application effectively solves the dimension interference and the excessive hard rejection problem in the multi-dimensional value alignment, improves the alignment accuracy, and maintains the general ability of the large language model.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing and artificial intelligence security alignment technology, and in particular relates to a multi-dimensional medical ethics value alignment method for causal large language models, specifically belonging to parameter-efficient fine-tuning (PEFT) and representation space structured geometric control technology. Background Technology

[0002] Large Language Models (LLMs) have demonstrated significant potential in high-risk applications such as medical clinical decision-making, automated medical record writing, and patient-centered interactive consultations in recent years. However, clinical medicine places extremely stringent ethical demands, far exceeding those of ordinary natural language generation tasks. Traditional alignment methods, such as Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO), typically simplify the alignment task to a one-dimensional "safe" or "harmless" control. However, in real-world clinical practice, sound medical decisions cannot be achieved simply by avoiding harmful outputs; rather, they require delicate trade-offs among multiple, even conflicting, ethical dimensions.

[0003] Typical medical ethical values ​​encompass at least several independent dimensions, including safety, autonomy, justice, integrity, and dignity and duty. This multi-dimensional and heterogeneous system of medical values ​​faces the following technical bottlenecks when applied to large language model alignment.

[0004] First, one-dimensional safety alignment leads to excessive hard rejection. Traditional safety alignment uses a single global safety vector constraint model, which makes it impossible to flexibly allocate alignment emphasis according to specific situations. When faced with clinical questioning on high-risk treatment plans or sensitive ethical topics, the model frequently generates large-scale, indiscriminate hard rejection, greatly reducing the practical value of medical big language models.

[0005] Second, value leakage and dimensional interference exist within the model's internal representation space. Existing research shows that directly intervening in the high-density representation layer of large language models can easily cause unintended activation or inhibition of non-target value dimensions, a phenomenon known as "value leakage." For example, when individually enhancing the model's altruistic or safety-oriented tendencies, unpredictable declines in professionalism or patient autonomy often occur. Current methods lack mathematically meaningful decoupling mechanisms between different value dimensions.

[0006] Third, catastrophic forgetting and degradation of general language capabilities. In order to achieve multi-dimensional value alignment at the same time, traditional multi-objective training usually fine-tunes the global parameters of the model, which can easily destroy the general capabilities already possessed by the large language model, such as logical reasoning, common sense question answering, and medical calculation, resulting in a zero-sum dilemma between alignment strength and general performance. Summary of the Invention

[0007] To address the aforementioned problems, this invention proposes a multidimensional low-rank subspace coordination and alignment method for medical large language models. This invention transforms multidimensional ethical values ​​from abstract semantic concepts into computable and projectible geometric low-rank subspaces within the representation space of the large language model. Furthermore, it introduces a cross-subspace orthogonal decoupling mechanism and a context-adaptive routing coordination network, thereby resolving the challenges of conflict trade-offs and mutual interference in multidimensional clinical value alignment without compromising the model's general capabilities.

[0008] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, the present invention implements a multidimensional value alignment method for a medical large language model based on low-rank multi-subspace, characterized by comprising the following steps: Step 1: Construct a multidimensional medical ethics value preference corpus. The corpus defines multiple different medical value dimensions and collects corresponding representative text samples for each medical value dimension. Step 2: Based on the corpus, construct multiple mutually orthogonal low-rank medical value subspaces. The number of the multiple subspaces is equal to the number of the multiple medical value dimensions, and the multiple subspaces correspond one-to-one with the multiple medical value dimensions. Step 3: For a given clinical input text, extract the hidden vector of the large language model in the specified intermediate layer, and project the hidden vector to the multiple low-rank medical value subspaces respectively to obtain the current value state vector in the multiple value dimensions. Step 4: Based on the hidden vectors of preferred and unpreferred answers, construct the human expected value optimization direction vector within the multiple low-rank medical value subspaces; Step 5: In the target parameter layer of the large language model, construct a low-rank parameter update module for each low-rank medical value subspace, which is subject to the geometric constraints of its corresponding subspace, so that the parameter update amount of the module is strictly limited to the corresponding subspace. Step 6: Construct a cross-subspace coordinated gating network to receive the global feature vector of the clinical input text, dynamically calculate the context weights of each value dimension, and perform a weighted summation of the outputs of each low-rank parameter update module to obtain the overall parameter correction amount for the current context. Step 7: Construct a joint loss function and fine-tune the trainable parameters of the low-rank parameter update module and the gating network end-to-end to achieve dynamic coordination and alignment of the large language model across multiple medical value dimensions.

[0009] In some implementations, step 2 specifically includes: Step 21, Feature Extraction and Matrix Construction: The large language model with frozen input parameters of the representative text sample is used to extract the hidden vectors of the specified intermediate layer and perform mean pooling to obtain the sentence-level semantic vectors corresponding to each value dimension; based on the sentence-level semantic vectors corresponding to each value dimension, the initial representation matrix is ​​constructed. Step 22, obtaining the single-dimensional low-rank basis matrix: perform singular value decomposition on the initial representation matrix, and extract the left singular vectors corresponding to the first few largest singular values ​​as the low-rank basis matrix for each value dimension; Step 23, Global Orthogonalization and Subspace Basis Determination: The low-rank basis matrices corresponding to the multiple value dimensions are concatenated column by column, and the concatenated matrix is ​​subjected to global QR decomposition to obtain an orthogonal matrix. The orthogonal matrix is ​​then divided into column vector groups with the same number as the multiple value dimensions, which serve as the basis for the multiple mutually orthogonal low-rank medical value subspaces.

[0010] In some implementations, in step S5, the parameter update amount of the low-rank parameter update module Determined by the following formula:

[0011] in, The orthogonalized low-rank basis matrix obtained in step S23 is frozen during training. It is a trainable low-rank coefficient matrix. is a trainable low-rank basis transpose matrix.

[0012] In some implementations, in step S6, the cross-subspace coordination gating network is a two-layer multilayer perceptron, whose output is normalized by the Softmax function to obtain the context weights for each value dimension. Their sum is 1.

[0013] In some implementations, the joint loss function in step S7 Determined by the following formula:

[0014] in, For language modeling loss, For the first Loss of directional consistency across value dimensions This is an orthogonal constraint penalty term. For KL divergence consistency loss, , , The weighting hyperparameters are used to balance the losses of each part.

[0015] In some implementations, the target parameter layer is the query vector projection layer in the self-attention mechanism of the large language model.

[0016] Secondly, this invention realizes a multidimensional value alignment system based on a low-rank multi-subspace medical large language model, comprising: The corpus construction module is used to construct a multidimensional medical ethics value preference corpus. The corpus defines multiple different medical value dimensions and collects corresponding representative text samples for each medical value dimension. The subspace construction module is used to construct multiple mutually orthogonal low-rank medical value subspaces based on the corpus set. The number of the multiple subspaces is equal to the number of the multiple medical value dimensions, and the multiple subspaces correspond one-to-one with the multiple medical value dimensions. The value state acquisition module is used to extract the hidden vector of the large language model in a specified intermediate layer for a given clinical input text, and project the hidden vector to the multiple low-rank medical value subspaces respectively to obtain the current value state vector in the multiple value dimensions. An optimization direction construction module is used to construct human expected value optimization direction vectors within the multiple low-rank medical value subspaces based on the hidden vectors of preferred and unpreferred answers. A low-rank parameter update module is set in the target parameter layer of the large language model. For each low-rank medical value subspace, a low-rank parameter update module is constructed, which is subject to the geometric constraints of its corresponding subspace, so that the parameter update amount of the module is strictly limited to the corresponding subspace. A cross-subspace coordinated gating network is used to receive the global feature vector of the clinical input text, dynamically calculate the context weight of each value dimension, and perform a weighted summation of the outputs of each low-rank parameter update module to obtain the overall parameter correction amount for the current context. The joint fine-tuning module is used to construct a joint loss function and perform end-to-end fine-tuning of the trainable parameters of the low-rank parameter update module and the gating network to achieve dynamic coordination and alignment of the large language model across multiple medical value dimensions.

[0017] In some implementations, the subspace construction module specifically includes: The feature extraction unit is used to extract the hidden vectors of the specified intermediate layer from the large language model with the input parameters of the representative text sample, and perform mean pooling to obtain the sentence-level semantic vectors corresponding to each value dimension, thus forming the initial representation matrix. The singular value decomposition unit is used to perform singular value decomposition on the initial representation matrix, and extract the left singular vectors corresponding to the first few largest singular values ​​as the low-rank basis matrix of the value dimension. The orthogonalization unit is used to concatenate the K low-rank basis matrices column-wise, perform global QR decomposition to obtain an orthogonal matrix, and cut out K column vectors from the orthogonal matrix as the basis of K mutually orthogonal low-rank medical value subspaces.

[0018] Thirdly, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the program, it implements the multidimensional value alignment method for a medical large language model based on low-rank multi-subspace as described in any one of the claims.

[0019] Fourthly, a computer-readable storage medium storing a computer program thereon, characterized in that, when executed by a processor, the program implements the multidimensional value alignment method for a medical large language model based on a low-rank multi-subspace as described in any one of the claims.

[0020] This invention addresses the technical solution of constructing multiple mutually orthogonal low-rank medical value subspaces, thereby mathematically severing the representational entanglement between multidimensional medical ethics and significantly reducing the unexpected activation of non-target values. Compared with existing technologies, this invention achieves remarkable technical advantages. 1. By constructing a low-rank medical value subspace that is orthogonal to multiple medical value dimensions, the different ethical dimensions are completely decoupled in the model's internal representation space, thereby significantly reducing the unexpected activation of non-target values ​​and improving the accuracy and stability of multi-dimensional value alignment.

[0021] 2. By extracting the intermediate hidden vectors of clinical input and projecting them onto each orthogonal subspace, and constructing optimization directions by combining the hidden vector differences of preferred and non-preferred responses, the abstract multidimensional ethical values ​​are transformed into computable and traceable geometric state vectors within the model. This upgrades the alignment target from a traditional static scalar to a dynamic direction vector, achieving refinement and quantification of the alignment process.

[0022] 3. By constructing a geometrically constrained low-rank parameter update module for each subspace in the target parameter layer, the parameter update amount is strictly limited to the corresponding subspace, eliminating the value drift in non-target regions from the root. At the same time, the number of parameters participating in gradient training accounts for only a very small part of the total number of model parameters, which greatly reduces the computational cost while achieving accurate alignment.

[0023] 4. By constructing a cross-subspace coordinated gating network and combining it with a joint loss function for end-to-end fine-tuning, we achieved adaptive adjustment of alignment weights for each value dimension according to the clinical context. This enables the generation of mild, professional, and ethically conscious responses in high-risk situations, while significantly improving the accuracy of multidimensional alignment and maintaining the general language capability and medical common sense capability of the large language model. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the overall process of the multidimensional value alignment method for a medical large language model based on low-rank multi-subspaces, as presented in this invention. Figure 2 This is a block diagram of the multidimensional value alignment system for a medical large language model based on low-rank subspaces, as described in this invention. (Low-rank subspace alignment schematic diagram) Figure 3 This is a flowchart illustrating the implementation of the multidimensional value alignment system method for a medical large language model based on low-rank multi-subspace of the present invention. Figure 4 The overall architecture diagram for low-rank multi-subspace modeling based on mean pooling, singular value decomposition (SVD) dimensionality reduction truncation, and global QR orthogonality decoupling is shown. Figure 5 A schematic diagram illustrating the forward flow and gradient calculation principles of the low-rank parameter update module constrained by subspace and the cross-subspace coordinated gating network. Figure 6 This is a comparative graph showing the experimental performance of the method of this invention and the baseline method in assessing medical safety rejection rate, overall accuracy of nursing values ​​assessment, and balance of ethical dimensions. Detailed Implementation

[0025] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0026] Implementation Figure 1 The flowchart of the multidimensional value alignment method for a medical large language model based on low-rank multi-subspace of the present invention is shown. The flowchart specifically includes the following steps: Step 1: Construct a multidimensional medical ethics value preference corpus. The corpus defines multiple different medical value dimensions and collects corresponding representative text samples for each medical value dimension. Step 2: Based on the corpus, construct multiple mutually orthogonal low-rank medical value subspaces. The number of the multiple subspaces is equal to the number of the multiple medical value dimensions, and the multiple subspaces correspond one-to-one with the multiple medical value dimensions. Step 3: For a given clinical input text, extract the hidden vector of the large language model in the specified intermediate layer, and project the hidden vector to the multiple low-rank medical value subspaces respectively to obtain the current value state vector in the multiple value dimensions. Step 4: Based on the hidden vectors of preferred and unpreferred answers, construct the human expected value optimization direction vector within the multiple low-rank medical value subspaces; Step 5: In the target parameter layer of the large language model, construct a low-rank parameter update module for each low-rank medical value subspace, which is subject to the geometric constraints of its corresponding subspace, so that the parameter update amount of the module is strictly limited to the corresponding subspace. Step 6: Construct a cross-subspace coordinated gating network to receive the global feature vector of the clinical input text, dynamically calculate the context weights of each value dimension, and perform a weighted summation of the outputs of each low-rank parameter update module to obtain the overall parameter correction amount for the current context. Step 7: Construct a joint loss function and fine-tune the trainable parameters of the low-rank parameter update module and the gating network end-to-end to achieve dynamic coordination and alignment of the large language model across multiple medical value dimensions.

[0027] like Figure 2 As shown, the present invention illustrates a multidimensional value alignment system for a medical large language model based on a low-rank multi-subspace, which includes the following modules connected in sequence: The corpus construction module 100 is used to construct a multidimensional medical ethics value preference corpus. The corpus defines multiple different medical value dimensions and collects corresponding representative text samples for each medical value dimension. The subspace construction module 200 is used to construct multiple mutually orthogonal low-rank medical value subspaces based on the corpus set. The number of the multiple subspaces is equal to the number of the multiple medical value dimensions, and the multiple subspaces correspond one-to-one with the multiple medical value dimensions. The value state acquisition module 300 is used to extract the hidden vector of the large language model in a specified intermediate layer for a given clinical input text, and project the hidden vector to the multiple low-rank medical value subspaces respectively to obtain the current value state vector in the multiple value dimensions. The optimization direction construction module 400 is used to construct the human expected value optimization direction vector in the plurality of low-rank medical value subspaces based on the hidden vectors of preferred and unpreferred answers. The low-rank parameter update module 500 is set in the target parameter layer of the large language model. It constructs a low-rank parameter update module for each low-rank medical value subspace, which is subject to the geometric constraints of its corresponding subspace, so that the parameter update amount of the module is strictly limited to the corresponding subspace. A cross-subspace coordination gating network 600 is used to receive the global feature vector of the clinical input text, dynamically calculate the context weights of each value dimension, and perform a weighted summation of the outputs of each low-rank parameter update module to obtain the overall parameter correction amount for the current context. The joint fine-tuning module 700 is used to construct a joint loss function and perform end-to-end fine-tuning of the trainable parameters of the low-rank parameter update module and the gating network to achieve dynamic coordination and alignment of the large language model across multiple medical value dimensions.

[0028] Furthermore, the subspace construction module 200 specifically includes: The feature extraction unit 210 is used to extract the hidden vector of the specified intermediate layer from the large language model with the input parameters of the representative text sample and perform mean pooling to obtain the sentence-level semantic vector corresponding to each value dimension, thus forming an initial representation matrix. The singular value decomposition unit 220 is used to perform singular value decomposition on the initial representation matrix and extract the left singular vectors corresponding to the first few largest singular values ​​as the low-rank basis matrix of the value dimension. The orthogonalization unit 230 is used to concatenate the K low-rank basis matrices column-wise, perform global QR decomposition to obtain an orthogonal matrix, and cut out K column vectors from the orthogonal matrix as the basis of K mutually orthogonal low-rank medical value subspaces.

[0029] like Figure 3 The diagram illustrates the complete processing flow from raw data input to the final generation of low-rank subspace basis. The entire process is divided into three core stages: feature extraction and initialization, single-dimensional low-rank basis construction, and global orthogonalization integration. From top to bottom, these include: Data and Dimension Input Phase: The implementation process begins with a "multidimensional medical value dataset," which specifically lists samples such as "safety, autonomy, and fairness." This raw data flows downwards to nodes in K value dimensions, establishing the number of objects to be processed and the classification boundaries.

[0030] Feature extraction and decomposition stage: This stage is labeled "learning the feature subspace for each value" and includes specific operational logic: Data processing flow: The process sequentially executes data organization -> model representation extraction -> main direction decomposition -> core low-rank feature extraction.

[0031] Technical implementation flow: This corresponds to specific mathematical operations, including inputting the problem -> processing the basic model -> obtaining mixed information -> projecting it onto a specific value space.

[0032] Basis generation stage: After the information is processed as described above, the final product is generated: a basis of K mutually orthogonal low-rank medical value subspaces.

[0033] This diagram illustrates the computational path that starts with raw medical data, processes it through modeling and feature decomposition, and ultimately establishes an orthogonal basis.

[0034] The specific technical solution of the present invention will be further described below with reference to specific examples: Specifically, based on the NurValues ​​nursing multidimensional preference dataset and five classic medical ethical values, a multidimensional medical ethical value preference corpus (i.e., K=5) was constructed. The five value dimensions are: ① Altruism (caring for the well-being of others), ② Human Dignity (respecting the dignity of every individual), ③ Integrity (honesty and integrity), ④ Justice (fairness and impartiality), and ⑤ Professionalism (adhering to professional norms). Representative text samples were collected for each value dimension, with each sample accompanied by two sets of data: one set of human-labeled preferred answers that conform to the specific ethical dimension, and the other set of non-preferred answers that deviate from the expected values ​​of that dimension. The backbone causal large language model used was Qwen3-8B and Llama-3.1-8B-Instruct, both of which are currently widely representative 8 billion parameter decoder architectures (Decoder-Only, i.e., Transformer structures consisting only of stacked decoder layers). The dataset is preprocessed to standardize spelling errors, remove linguistic errors and punctuation marks such as commas and quotation marks, and remove illegal characters such as "ufeff". Finally, the corpus is divided into training set and test set.

[0035] (2) Construct a low-rank subspace representation model based on singular value decomposition (SVD) truncation and global QR orthogonalization, extract features from K value dimensions respectively, and construct mutually orthogonal low-rank medical value subspaces, as follows: Step 1: Select the Query Projection (q_proj) layer in the self-attention mechanism of the large language model as the target layer for feature extraction and fine-tuning intervention. Here, q_proj is not a function, but rather the name of the linear layer corresponding to the weight matrix used in the Transformer self-attention module to linearly map the hidden state to the query vector. The size of its weight matrix is ​​d_model × d_model (4096 × 4096 in this embodiment). The reason for selecting this layer is that modern large language models generally adopt the Grouped Query Attention (GQA) structure; if the Value Projection (v_proj) layer is selected as the intervention layer, the subspace dimension will be truncated and the representation will fail due to the compression of the dimension of this layer. Therefore, this invention precisely limits the intervention target layer to the q_proj layer.

[0036] In practice, 64 clinical nursing scenario question-answer pairs are extracted for each value dimension to form a calibration sample set; and the q_proj linear layer at layer 20 in the backbone model's self-attention mechanism is positioned as the specific operation object. When the calibration text propagates forward, the activation tensor output by the target layer is extracted, and mean pooling is performed along the sequence length to obtain the sentence-level semantic vector for each sample. In this embodiment, this vector belongs to a 4096-dimensional real space (i.e., each sentence is represented as a vector of length 4096). The 64 sample vectors of the same value dimension are stacked column-wise to form an initial representation matrix of size 4096×64.

[0037] Step 2: Input representative text samples into the causal large language model with frozen parameters, and extract the activation tensor output by the q_proj layer for each sample. Apply an attention mask (attention_mask, a labeled array used to mask padding positions) to perform mean pooling along the sequence length to obtain the sentence-level semantic vector for each sample. The calculation formula is as follows:

[0038] The numerator is the sum of the hidden vectors at all real word positions, and the denominator is the total number of real word positions. Dividing these two values ​​yields the average vector. All sentence-level vectors for the same value dimension are stacked column-wise to form the initial representation matrix for that dimension. .

[0039] Step 3: Perform column normalization on the initial representation matrix, that is, subtract the mean of each column from the mean of that column. μ_kThen divide by the sum of the standard deviation of that column and a very small positive constant (to prevent division by zero) to eliminate scale differences between different samples. Then perform Singular Value Decomposition (SVD) on the normalized initial representation matrix:

[0040] In the formula, This is a left singular vector matrix, where each column represents a principal direction. It is a singular value diagonal matrix, where each diagonal element represents the importance of that direction. This is the transpose of the right singular vector matrix. (Truncation) Center front The left singular vector corresponding to the largest singular value is used as the th largest singular value. k A low-rank basis matrix of each value dimension is used to transform abstract values ​​into a computable geometric subspace within a large language model.

[0041] Step 4: To completely sever the entanglement of features among the five major medical ethical values ​​within the large language model, a global orthogonalization is performed on all basis matrices. Specifically, the five low-rank basis matrices (each 4096×8) in this embodiment are horizontally concatenated column-wise to form a global projection basis matrix of size 4096×40; then, a global QR decomposition is performed on it.

[0042] in, Q It is an orthogonal matrix (the column vectors are completely perpendicular to each other). R This is an upper triangular matrix (all elements below its diagonal are zero). The resulting orthogonal matrix is ​​then extracted. Q A column vector (4096×40) is divided into K segments (5 segments in this embodiment) by splitting it into 8 columns each. The columns then form a new basis matrix of mutually orthogonal value subspaces (each of which is 4096×8), denoted as The basis matrix is ​​orthogonalized into a low-rank medical value subspace. This basis matrix is ​​then registered as a non-trainable constant (its value remains fixed during training) in the model's hidden buffer, preventing it from participating in gradient updates throughout the fine-tuning process. This orthogonalization process mathematically eliminates interference between different value dimensions, fundamentally avoiding value leakage during alignment.

[0043] (3) For any clinical input x Extract the hidden feature vectors of the specified intermediate layer and project them onto K orthogonalized medical value subspaces. Specifically, define the... k Global orthogonal projection operator on each subspace It is given by the product of the basis matrix of the subspace and its transpose (i.e. Applying the projection operator to the intermediate hidden vector of the clinical input yields the result of the input at the [missing information - likely a specific location or event]. k The current value state vector in each value dimension is used to characterize the relative position and activation intensity of the large language model in that value dimension in the representation space using high-dimensional coordinates.

[0044] (4) Constructing human expected value optimization directions based on the comparison of preferences and non-preferences. For the same clinical question, extract the intermediate hidden vectors of preferred answers that meet human expectations and non-preference answers that do not meet human expectations from the corpus at a specified intermediate layer (output of q_proj at layer 20); subtract the two vectors, and the resulting difference vector is the human expected value optimization direction vector in each low-rank medical value subspace, denoted as . d k (x). Where the subscript k represents the k-th dimension of medical value, and the superscript... This vector represents the desired optimization direction as expected by humans. The 'x' within the parentheses represents the current clinical input text, meaning that this direction vector varies depending on the input context. In geometric space, this direction vector explicitly indicates the specific direction in which the large language model should move to better align with human expectations for this ethical dimension. This avoids the limitations of traditional alignment, which simplifies value preferences into static good / bad scalars, and better reflects the direction that should be adjusted to meet human expectations for this medical ethical dimension.

[0045] (5) In the target parameter layer of the large language model (referring to the weight matrix of the q_proj layer in this invention), for the th k Each value dimension constructs a low-rank parameter update module subject to the geometric constraints of its corresponding orthogonal subspace. The formula for the parameter update amount is:

[0046] in, Always keep frozen, only for trainable low-rank coefficient matrices. With the transpose of the low-rank basis Perform gradient updates. Since the parameter update amount is always determined by... Since the linear representation means that all parameter updates generated by the fine-tuning layer must always fall within the orthogonal subspace, thus achieving zero interference with irrelevant dimensions and fundamentally eliminating value drift in non-target regions.

[0047] (6) Construct a cross-subspace coordinated gating network. The gating network consists of two fully connected layers, a Rectified Linear Unit (ReLU) activation function, and a Softmax normalization layer. The workflow is as follows: It receives the global sentence-level hidden representation of the input clinical question, first reduces the dimensionality to an intermediate latent space through the first fully connected layer, then activates it with ReLU, and finally maps it to a logarithmic vector containing K scalar components through the second fully connected layer; finally, it normalizes the logarithmic vector into contextual weights for each value dimension through the Softmax function.

[0048] The resulting normalized weights are non-negative and sum to 1. These weights are then used as contextual weights to perform a weighted summation of the outputs of each low-rank parameter update module, thus obtaining the weights specific to the current clinical input. x The overall parameter correction amount. This dynamic fusion method enables the large language model to automatically increase the weight of safety and responsibility when facing emergency rescue situations, and automatically tilt towards the patient dignity and autonomy subspace when facing mitigating medical situations, demonstrating good multi-ethical adaptive balancing ability.

[0049] (7) The joint loss consists of four parts, and the joint multi-task optimization loss function is constructed as shown in the following equation:

[0050] in, The autoregressive cross-entropy language modeling loss is used for large language models to maintain the language generation ability of the original model. For the first k The cosine direction consistency loss over each subspace is defined as the inverse of the cosine similarity between the model's current projection vector in that subspace and the human-expected optimization direction vector obtained in step (4). Minimizing this loss drives the large language model representation to move along the direction of human preference. L ortho To provide an adaptive orthogonality penalty, the Frobenius norm is used to penalize the non-orthogonal parts between different subspaces, thereby monitoring and maintaining the orthogonality of the space. L KL The KL divergence consistency loss ensures that the output probability distribution of the model after fine-tuning does not deviate from the original model. α , β , γThese are three hyperparameters used to balance the relative importance of the various loss components. During fine-tuning training, all original backbone weights of the causal large language model are frozen, and end-to-end gradient backpropagation updates are performed only on the low-rank coefficient matrix, low-rank basis, and gating network parameters using the AdamW optimizer (Adam with WeightDecay, an adaptive moment estimator with weight decay).

[0051] like Figure 4 The figure illustrates the overall architecture of mean pooling, SVD dimensionality reduction, and global QR orthogonalization. It shows the overall architecture for low-rank multi-subspace modeling based on mean pooling, SVD dimensionality reduction truncation, and global QR orthogonalization decoupling. The specific process is as follows: Feature extraction and mean pooling: The large language model with the input parameters of representative text samples is frozen, the hidden vectors of the specified intermediate layer (such as the q_proj layer) are extracted, and mean pooling is performed using attention masks to obtain the sentence-level semantic vector of each sample, which forms the initial representation matrix.

[0052] SVD dimensionality reduction truncation: Perform singular value decomposition on the initial representation matrix of each value dimension, and truncate the left singular vectors corresponding to the first few largest singular values ​​as the low-rank basis matrix of that dimension, thus achieving preliminary dimensionality reduction from high-dimensional representation to low-dimensional subspace.

[0053] Global QR orthogonal decoupling: Concatenate the K low-rank basis matrices column-wise and perform global QR decomposition to obtain an orthogonal matrix Q. Extract K column vectors from Q to serve as the basis of a mutually orthogonal low-rank medical value subspace, thus mathematically severing the representational entanglement between different value dimensions.

[0054] This architecture transforms abstract, multidimensional medical ethics values ​​into a computable and projectible geometric low-rank subspace within a large language model, providing a structured foundation for subsequent context-adaptive alignment.

[0055] like Figure 5 As shown, the forward and gradient calculation principles of the low-rank parameter update module and the gated network are illustrated.

[0056] like Figure 6 The figure shows the final comparison results between the method of the present invention and the baseline model on multiple evaluation indicators. This figure visually illustrates the comparison effect between the method of the present invention and the baseline method on three key indicators, as detailed below: 1. Safety and accuracy metrics (bar chart above) The MedSafety rejection rate assessment demonstrates the model's ability to identify and block potentially unsafe medical advice requests or unethical, misleading instructions. For the Llama model, the rejection rate before alignment was 70.2%, which increased to 97.1% after alignment, an absolute improvement of 26.9 percentage points. For the Qwen model, the rejection rate before alignment was 62.0%, which increased to 96.6% after alignment, an absolute improvement of 34.6 percentage points. The rejection rates for both models remained stable above 96% after alignment, and the gap between the two backbone models converged from 8.2 percentage points before alignment to 0.5 percentage points. This indicates that the method described in this invention can significantly and consistently enhance the model's medical safety defense capabilities, and this gain is independent of any specific backbone model architecture.

[0057] The NurValues ​​overall accuracy assessment demonstrates the model's understanding and adherence to nursing ethical values. The overall accuracy of the Llama model improved from 91.0% before alignment to 96.1%, and the Qwen model improved from 92.2% to 93.5%. Given that the baseline accuracy was already above 90% before alignment, this invention still achieved absolute improvements of 5.1 and 1.3 percentage points respectively, equivalent to reducing the residual false positive rate by approximately 50% and 15%, respectively. This result, coupled with the significant improvement in the rejection rate, indicates that this invention, while enhancing the ability to safely reject cases, did not sacrifice the model's accuracy in discriminating nursing ethical situations; the two exhibit a synergistic improvement relationship, rather than the inverse relationship commonly seen in traditional one-dimensional safe alignment.

[0058] 2. Ethical Dimension Balance Indicators (Radar Chart Below) The Llama model's ethical dimension balance diagram compares the performance of the model before alignment (light-filled areas) and after alignment (dark solid-line outlines) across five value dimensions: professionalism, altruism, integrity, justice, and human dignity. Before alignment, the five-dimensional outlines show significant inward contraction in the directions of justice and human dignity, with an overall irregular shape, indicating an imbalance in the development of each value dimension. After alignment, the outlines expand outward and approach a regular pentagon, with the values ​​of the five dimensions becoming more consistent, indicating a more balanced distribution of the model's intrinsic values. This verifies the inhibitory effect of orthogonal subspace decoupling and gating coordination mechanisms on dimensional interference.

[0059] The Qwen model's ethical dimension balance diagram also compares the performance before alignment (light-filled areas) and after alignment (dark solid-line outlines). Before alignment, there are obvious dips in the altruistic and fairness directions, exhibiting a typical bias towards certain value dimensions. After alignment, the outline expands significantly and the shape becomes more regular and symmetrical, with a significant narrowing of the differences in values ​​across dimensions. This indicates that the invention successfully corrected the model's shortcomings in some ethical dimensions, achieving a coordinated and balanced alignment of multi-dimensional values.

[0060] In summary, this graph demonstrates through quantitative data the effectiveness of this invention in improving the accuracy of medical safety protection and ethical value in large language models, as well as maintaining multi-dimensional balanced development.

[0061] Among them, the current clinical input x Extract the intermediate hidden vector of the target layer q_proj at layer 20 along the forward path; multiply this vector on the left by the projection operators of the five orthogonal subspaces (the projection operators are given by the product of the basis matrix of the subspace and its transpose) to obtain the input. x The current value state vectors in the five value dimensions respectively characterize the activation intensity and geometric position of the large language model in the five dimensions of altruism, human dignity, honesty and trustworthiness, social justice and professional responsibility.

[0062] Specifically, a low-rank parameter update module constrained by subspace geometry, designed according to this invention, is inserted into the q_proj layer (4096×4096) of the self-attention mechanism of the large language model. This module is named ValueAlignedLoRA (i.e., the value-aligned low-rank adaptation module). In this embodiment, the low-rank intermediate rank... r lora The value is set to 8; where the subscript lora indicates that this rank is the intermediate rank within the low-rank parameter update module described in this invention, that is, the rank number of the low-rank decomposition in each value subspace. Its value determines the scale of trainable parameters for this module; the smaller the value, the fewer trainable parameters and the smaller the perturbation to the original model. At the start of fine-tuning, the trainable low-rank coefficient matrix... (Size 8×8) Initialized randomly using a uniform Kaiming distribution; low-rank basis transpose matrix. (Size 4096×8) It adopts all-zero initialization, thus ensuring that the parameter update is always equal to zero at the moment of fine-tuning. The model can completely inherit the attention path of the original model in the q_proj layer, eliminating initialization perturbation.

[0063] The project constructs a lightweight gated routing module, named CrossSubspaceGating, consisting of two fully connected layers, a ReLU activation function, and a Softmax normalization layer. The forward computation logic is as follows: It receives the 4096-dimensional global sentence-level hidden representation of the current problem, reduces it to a 1024-dimensional compressed latent space through the first fully connected layer, applies the ReLU activation function (whose output is the larger of the input and zero), and then maps it to a 5-dimensional logarithmic component space through the second fully connected layer. Finally, it normalizes the representation using the Softmax function to obtain five context-specific gate weights (the sum of these five weights is 1). In forward inference, the gated routing network outputs fine-tuned superimposed weights for each subspace based on the specific input context. These weights are then weighted and summed with the outputs of each low-rank parameter update module to obtain the weights specific to the current input. xThe overall parameter correction amount. This correction amount is superimposed on the original weights during the model's forward computation, and is combined with a preset low-rank scaling factor (…). α lora / r lora ,in α lora Scaling factor for low-rank update quantity r lora For the aforementioned low-rank intermediate rank, in this embodiment we take 32 and 8 respectively, and divide them to get the scaling factor 4.0. This, together with the original q_proj layer, determines the final forward activation output of this layer.

[0064] The joint multi-task optimization loss function is constructed by a weighted mixture of cross-entropy language modeling loss, cosine consistency direction loss, orthogonality constraint penalty term, and KL divergence consistency loss. In joint fine-tuning, the batch size (number of samples processed simultaneously in a single run) is set to 2, the gradient accumulation step size is set to 4 to obtain stable gradients, and the equivalent batch size is 8. The AdamW optimizer is used, with an initial learning rate of 1×10⁻⁶. -4 The low-rank coefficient matrix, low-rank basis, and gating network parameters are jointly trained end-to-end for 5 epochs (one epoch is the process of going through the entire training set once). During backpropagation, all original backbone weights of the large language model are frozen, and gradient updates are performed only on the low-rank trainable parameters and gating network parameters.

[0065] To accurately assess the alignment level of the fine-tuned large language model across five core ethical values ​​in healthcare, this embodiment designs a deterministic zero-shot (zero-sample, i.e., evaluation without providing any examples) automated evaluation process. This evaluation process achieves deterministic closed-loop evaluation through three mechanisms: structured ethical evaluation cue word engineering, high-throughput inference control and decoding protection, and two-layer fault-tolerant classification text parsing, as detailed below: Firstly, the structured ethical assessment prompt word engineering: For each nursing scenario text in the test set and its corresponding core value dimension to be evaluated, the system dynamically constructs a formatted single-round zero-sample assessment prompt word template. This prompt word assigns the nursing ethics assessment expert role to the large language model and explicitly provides strict output space definitions for aligned and misaligned behavior to constrain free generation behavior.

[0066] Secondly, high-throughput inference control and decoding protection: During the inference generation stage of the large language model, greedy decoding (i.e., selecting the word with the highest probability at each step) is enforced, limiting the maximum generation length per iteration to 10 words to ensure high reproducibility and determinism of the output in multiple evaluations. For next-generation inference-oriented large language models with a long text chain-of-thought (CoT) inference path, the chain-of-thought generation switch is explicitly turned off when calling dialogue templates to construct input, thereby obtaining concise label predictions and significantly reducing the computational latency and inference cost caused by meaningless words.

[0067] Thirdly, a two-layer fault-tolerant classification text parsing: Since large language models may output non-standard format responses containing redundant punctuation, articles, or explanatory information during inference, this invention employs a cascaded two-layer fault-tolerant classification parsing logic. The first layer is precise substring matching, which converts the model-generated responses to lowercase and removes leading and trailing spaces, then searches for the presence of "misaligned" or "aligned," classifying them as either violations or alignment tags, respectively. The second layer is a fallback negative sensitive word scan. When the first-layer precise matching fails due to the generalization of the large language model's syntax, it automatically searches the entire response for predefined sets of strong negation and violation feature words (such as "not," "no," "violat," "against," "breach," "inappropriate," etc.). If any sensitive word is detected, the predicted value is robustly normalized to a violation; if none are detected, it is normalized to alignment by default. This two-layer parsing algorithm can effectively resolve the problem of irregular text output by large language models in zero-shot scenarios.

[0068] After analysis, the evaluation system calculates two main indicators on the global scale and the five major dimensions: overall accuracy and macro-F1 score (a comprehensive evaluation indicator that considers both precision and recall and takes the arithmetic mean of each category). This fine-grained, objective quantitative evaluation accurately reveals the model's precise control over heterogeneous values ​​such as social justice and autonomy.

[0069] The method of this invention was compared with a baseline model, and statistical indicators such as accuracy, macro F1, value leakage, and general capability retention rate were analyzed. The results are as follows: Figure 6 As shown, it can be intuitively observed that while the method of the present invention significantly improves the alignment accuracy of multidimensional medical ethics, the amount of unexpected activation of non-target values ​​is greatly reduced, and the general language ability and medical common sense ability remain basically stable without significant fluctuations, and catastrophic amnesia is basically not observed.

[0070] The above description is merely an embodiment of the present invention and is not intended to limit the methods proposed in this invention. The scope of protection of this invention is defined by the claims. Any obvious modifications or variations in form and detail made by those skilled in the art without departing from the spirit and scope of this invention should fall within the scope of protection of this invention.

Claims

1. A multidimensional value alignment method for a medical large language model based on low-rank multi-subspace, characterized in that, Includes the following steps: Step 1: Construct a multidimensional medical ethics value preference corpus. The corpus defines multiple different medical value dimensions and collects corresponding representative text samples for each medical value dimension. Step 2: Based on the corpus, construct multiple mutually orthogonal low-rank medical value subspaces. The number of the multiple subspaces is equal to the number of the multiple medical value dimensions, and the multiple subspaces correspond one-to-one with the multiple medical value dimensions. Step 3: For a given clinical input text, extract the hidden vector of the large language model in the specified intermediate layer, and project the hidden vector to the multiple low-rank medical value subspaces respectively to obtain the current value state vector in the multiple value dimensions. Step 4: Based on the hidden vectors of preferred and unpreferred answers, construct the human expected value optimization direction vector within the multiple low-rank medical value subspaces; Step 5: In the target parameter layer of the large language model, construct a low-rank parameter update module for each low-rank medical value subspace, which is subject to the geometric constraints of its corresponding subspace, so that the parameter update amount of the module is strictly limited to the corresponding subspace. Step 6: Construct a cross-subspace coordinated gating network to receive the global feature vector of the clinical input text, dynamically calculate the context weights of each value dimension, and perform a weighted summation of the outputs of each low-rank parameter update module to obtain the overall parameter correction amount for the current context. Step 7: Construct a joint loss function and fine-tune the trainable parameters of the low-rank parameter update module and the gating network end-to-end to achieve dynamic coordination and alignment of the large language model across multiple medical value dimensions.

2. The multidimensional value alignment method for a medical large language model based on low-rank multi-subspace as described in claim 1, characterized in that, Step 2 specifically includes: Step 21, Feature Extraction and Matrix Construction: The large language model with frozen input parameters of the representative text sample is used to extract the hidden vectors of the specified intermediate layer and perform mean pooling to obtain the sentence-level semantic vectors corresponding to each value dimension; based on the sentence-level semantic vectors corresponding to each value dimension, the initial representation matrix is ​​constructed. Step 22, obtaining the single-dimensional low-rank basis matrix: perform singular value decomposition on the initial representation matrix, and extract the left singular vectors corresponding to the first few largest singular values ​​as the low-rank basis matrix for each value dimension; Step 23, Global Orthogonalization and Subspace Basis Determination: The low-rank basis matrices corresponding to the multiple value dimensions are concatenated column by column, and the concatenated matrix is ​​subjected to global QR decomposition to obtain an orthogonal matrix. The orthogonal matrix is ​​then divided into column vector groups with the same number as the multiple value dimensions, which serve as the basis for the multiple mutually orthogonal low-rank medical value subspaces.

3. The multidimensional value alignment method for a medical large language model based on low-rank multi-subspace as described in claim 1, characterized in that, In step S5, the parameter update amount of the low-rank parameter update module Determined by the following formula: in, The orthogonalized low-rank basis matrix obtained in step S23 is frozen during training. It is a trainable low-rank coefficient matrix. is a trainable low-rank basis transpose matrix.

4. The multidimensional value alignment method for a medical large language model based on low-rank multi-subspace as described in claim 1, characterized in that, In step S6, the cross-subspace coordination gating network is a two-layer multilayer perceptron, and its output is normalized by the Softmax function to obtain the context weights of each value dimension. Their sum is 1.

5. The multidimensional value alignment method for a medical large language model based on low-rank multi-subspace as described in claim 1, characterized in that, The joint loss function in step S7 Determined by the following formula: in, For language modeling loss, For the first Loss of directional consistency across value dimensions This is an orthogonal constraint penalty term. For KL divergence consistency loss, , , The weighting hyperparameters are used to balance the losses of each part.

6. The multidimensional value alignment method for a medical large language model based on low-rank multi-subspace as described in claim 1, characterized in that, The target parameter layer is the query vector projection layer in the self-attention mechanism of the large language model.

7. A multidimensional value alignment system for a medical large language model based on low-rank multi-subspace, characterized in that, include: The corpus construction module is used to construct a multidimensional medical ethics value preference corpus. The corpus defines multiple different medical value dimensions and collects corresponding representative text samples for each medical value dimension. The subspace construction module is used to construct multiple mutually orthogonal low-rank medical value subspaces based on the corpus set. The number of the multiple subspaces is equal to the number of the multiple medical value dimensions, and the multiple subspaces correspond one-to-one with the multiple medical value dimensions. The value state acquisition module is used to extract the hidden vector of the large language model in a specified intermediate layer for a given clinical input text, and project the hidden vector to the multiple low-rank medical value subspaces respectively to obtain the current value state vector in the multiple value dimensions. An optimization direction construction module is used to construct human expected value optimization direction vectors within the multiple low-rank medical value subspaces based on the hidden vectors of preferred and unpreferred answers. A low-rank parameter update module is set in the target parameter layer of the large language model. For each low-rank medical value subspace, a low-rank parameter update module is constructed, which is subject to the geometric constraints of its corresponding subspace, so that the parameter update amount of the module is strictly limited to the corresponding subspace. A cross-subspace coordinated gating network is used to receive the global feature vector of the clinical input text, dynamically calculate the context weight of each value dimension, and perform a weighted summation of the outputs of each low-rank parameter update module to obtain the overall parameter correction amount for the current context. The joint fine-tuning module is used to construct a joint loss function and perform end-to-end fine-tuning of the trainable parameters of the low-rank parameter update module and the gating network to achieve dynamic coordination and alignment of the large language model across multiple medical value dimensions.

8. The multidimensional value alignment system for a medical large language model based on low-rank multi-subspace as described in claim 7, characterized in that, The subspace construction module specifically includes: The feature extraction unit is used to extract the hidden vectors of the specified intermediate layer from the large language model with the input parameters of the representative text sample, and perform mean pooling to obtain the sentence-level semantic vectors corresponding to each value dimension, thus forming the initial representation matrix. The singular value decomposition unit is used to perform singular value decomposition on the initial representation matrix, and extract the left singular vectors corresponding to the first few largest singular values ​​as the low-rank basis matrix of the value dimension. The orthogonalization unit is used to concatenate the K low-rank basis matrices column-wise, perform global QR decomposition to obtain an orthogonal matrix, and cut out K column vectors from the orthogonal matrix as the basis of K mutually orthogonal low-rank medical value subspaces.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the multidimensional value alignment method for medical large language models based on low-rank multi-subspace as described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the multidimensional value alignment method for medical large language models based on low-rank multi-subspace as described in any one of claims 1 to 6.

Images