An insurance contract interpretation large model core knowledge tamper-proofing method and system

Abstract

This invention discloses a method and system for preventing tampering with core knowledge in a large-scale insurance contract interpretation model. This method maintains consistency in core knowledge such as clause interpretation, standard Q&A, and claims procedures during continuous fine-tuning or domain adaptation. First, sentinel samples are constructed and baseline outputs are obtained. The importance of parameters at each layer of the model is calculated based on gradient sensitivity. Under parameter budget constraints, singular value decomposition is performed on the weight matrices of each layer, constructing a high-rank protection subspace and a low-rank update subspace to isolate the direction of core knowledge expression from the updatable direction. During the training phase, parameter updates are performed only in the low-rank subspace, and backtracking or adjustment is triggered by sentinel output consistency verification, thereby reducing the impact of model updates on core knowledge. This method requires no additional annotation and does not change the model structure, improving the stability and controllability of model updates in insurance contract interpretation scenarios.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of artificial intelligence and knowledge management in the insurance industry, and in particular to a method and system for preventing tampering with the core knowledge of a large-scale insurance contract interpretation model. Background Technology

[0002] In insurance business scenarios, information such as policy interpretation, claims processes, business rules, and financial statements typically require high stability and consistency. As the application of large-scale pre-trained language models in tasks such as insurance question answering, policy interpretation, and business text understanding continues to expand, continuous fine-tuning or industry-specific data adaptation is often necessary to improve the model's adaptability to specific business scenarios. However, due to variations in training data distribution, gradient direction conflicts between different tasks, and interference from outlier or malicious samples, the model is prone to deviating from established insurance core knowledge during updates, leading to output results that deviate from predetermined standards and impacting business accuracy and compliance.

[0003] Existing technologies typically employ methods such as freezing some parameters, adding network branches, or introducing additional regularization to maintain existing knowledge. However, these methods lack interpretable quantification of the importance of parameters at different layers and struggle to precisely constrain update directions that might disrupt core knowledge. Furthermore, in the highly stable knowledge scenarios of the insurance industry, these methods often fail to balance controllable model updates, adaptive training, and engineering feasibility, thus failing to meet the stability, reliability, and traceability requirements of actual business operations. Summary of the Invention

[0004] Purpose of the Invention: This invention aims to provide a method and system for preventing tampering with the core knowledge of a large-scale insurance contract interpretation model. By introducing sentinel sample sensitivity analysis, hierarchical importance allocation, and subspace projection update mechanism, the model adapts to new insurance tasks or new data distributions while reducing the disturbance to key knowledge representation during the update process, thereby improving the consistency and stability of the model's output in insurance contract interpretation scenarios.

[0005] Technical Solution: To achieve the above objectives, this invention proposes a method and system for preventing tampering with the core knowledge of a large-scale insurance contract interpretation model. This method is designed around the overall concept of "identifying key knowledge expression directions—constructing a protected parameter subspace—constraining parameter update paths," and mainly includes steps such as sentinel sample management, hierarchical sensitivity calculation, hierarchical importance fusion and rank-preserving allocation, subspace construction, and gradient dual projection update.

[0006] The specific plan is as follows:

[0007] Firstly, the present invention provides a method for preventing tampering with the core knowledge of a large-scale insurance contract interpretation model, comprising the following steps:

[0008] (1) Sentinel sample construction and baseline output acquisition: Based on the core knowledge of the interpretation of insurance terms, standard Q&A, claims process and financial standards, a set of sentinel samples is constructed; the input, standard output and evaluation method of each sentinel sample are recorded; the sentinel sample is input into the current base model to perform forward inference and obtain the corresponding baseline output, which is used for subsequent sensitivity calculation and consistency verification.

[0009] (2) Jacobi-based hierarchical sensitivity calculation: The sentinel samples are sequentially input into the base model to construct binary mask / label targets corresponding to key output positions, and backpropagation is performed on the label targets to obtain the gradient tensors of the weight matrices of each layer; the sensitivity index of each layer is calculated based on the norm or equivalent magnitude index of the gradient tensor, and the sensitivity indexes of all sentinel samples are aggregated to obtain the average sensitivity sequence arranged by layer;

[0010] (3) Layer importance fusion and rank retention allocation: The average sensitivity sequence is normalized; the structural importance of each layer is evaluated based on the similarity between the input features and output features of each layer; the normalized sensitivity and structural importance are fused according to preset weights to obtain the fused importance of each layer, and the corresponding retention ratio or retention dimension is assigned to each layer according to the fused importance, which is used to control the construction size of the high-rank subspace of each layer in the subsequent singular value decomposition.

[0011] (4) Subspace construction based on singular value decomposition: Perform singular value decomposition on the weight matrix of each layer; according to the retention ratio or retention dimension obtained in step (3), select the corresponding number of singular values ​​and their singular vectors to form the high-rank subspace of the layer, and form the low-rank subspace with the remaining singular vectors, so that the high-rank subspace and the low-rank subspace are linearly orthogonal, thereby completing the subspace partitioning of the layer.

[0012] (5) Gradient Dual Projection and Robust Update: After calculating the gradient of the loss function relative to the weight matrix of each layer in the subsequent training stage, the components of the gradient in the high-rank subspace direction are first removed, and then the remaining gradient is projected to the low-rank subspace through the projection operator to obtain the update gradient in the low-rank direction. Based on the update gradient, the corresponding weight parameters are updated only in the low-rank subspace direction, thereby reducing the disturbance of the knowledge representation direction in the high-rank subspace. During the training process, sentinel samples are periodically input into the model and compared with the baseline output to perform backoff, learning rate adjustment or retention rate adjustment when the output deviates greatly, so as to maintain the stability of the model in the direction of insurance core knowledge.

[0013] Further, step (1) includes the following sub-steps:

[0014] S1.1 Knowledge sorting and source determination: Based on the knowledge content in insurance business that needs to remain stable over a long period of time and should not be disturbed by the training process, the knowledge sources constituting the sentinel samples are determined. The knowledge content includes knowledge of clause interpretation, knowledge of standards and norms, fixed business rules, procedural knowledge, and question and answer information with unique output, which is used to clarify the semantic scope and expected output form of the sentinel samples.

[0015] S1.2 Sample Design and Standard Output Construction: Based on the aforementioned knowledge sources, an input representation capable of triggering the model to generate stable output is designed for each knowledge point, and a corresponding standard output is set for each sentinel sample; the input is in the form of natural language description and / or structured input, and the standard output is fixed text, structured information, and / or a numerical range that can be determined according to rules; and multiple semantically equivalent input representations can be designed for the same knowledge point to enhance coverage;

[0016] S1.3 Sample Normalization and Storage: The constructed sentinel samples are normalized to include at least sample identifier, input content, standard output, knowledge category identifier, evaluation method and evaluation threshold parameters. The normalized sample set is managed according to version and stored in a storage medium accessible to the training process so that it can be reused in different training tasks or training stages.

[0017] S1.4 Sample Screening and Evaluation Method Setting: Candidate sentinel samples are screened to ensure that they meet the preset requirements in terms of business criticality, semantic clarity, output uniqueness and evaluability; and according to different knowledge categories, at least one evaluation method is selected from exact matching, template matching, tolerance range judgment, structural consistency judgment and semantic similarity judgment, and corresponding thresholds or rule parameters are set for the evaluation method to monitor the consistency of model output during training.

[0018] S1.5 Baseline Output Acquisition: Perform forward inference on the sentinel sample using the current pedestal model, and record the model output as the baseline output of the sentinel sample; and record the model version information and configuration parameters related to inference; do not perform backpropagation on the sentinel sample during subsequent training; the baseline output serves as a reference benchmark for sensitivity calculation and consistency verification, and is used to determine the retention status of the core insurance knowledge in the model during the update process.

[0019] Furthermore, step (2) includes the following sub-steps:

[0020] S2.1 Forward Propagation and Feature Recording: Input the sentinel samples into the base model to perform forward propagation, obtain the corresponding model output, and cache intermediate activation information at a preset layer to support subsequent gradient calculation;

[0021] S2.2 Scalar Target Construction: For output positions that need to be kept stable, construct a binary mask vector aligned with the model output, and define a scalar target based on the binary mask vector to extract key output components, so that backpropagation only focuses on the key output positions;

[0022] S2.3 Backpropagation and Layered Gradient Calculation: Perform backpropagation on the scalar objective to obtain the gradient tensor corresponding to the weight matrix of each layer; wherein, the gradient tensor is mathematically equivalent to outputting the weighted aggregation term of the Jacobian matrix with respect to the weights at the mask position, thus eliminating the need to explicitly construct the complete Jacobian matrix;

[0023] S2.4 Sensitivity Measurement and Sample Aggregation: The Frobenius norm of the gradient tensor is used as the hierarchical sensitivity index of a single sentinel sample, and the hierarchical sensitivity index of all sentinel samples in the same layer is averaged to obtain the hierarchical average sensitivity, which is used for the layer importance fusion and quota allocation in step (3).

[0024] Furthermore, step (3) includes the following sub-steps:

[0025] S3.1 Sensitivity Normalization: In order to eliminate the influence of differences in parameter scale, activation distribution and gradient magnitude of different layers on the sensitivity measurement, the average sensitivity of the layers obtained in step (2) is linearly normalized to obtain the normalized sensitivity of each layer.

[0026] S3.2, Calculation and Normalization of Structural Layer Importance: The intermediate features obtained from the first forward propagation are used as input information for representation; for the... The cosine similarity of the input and output features of a layer is calculated and averaged over the batch dimension to obtain the structural importance index of that layer; the structural importance index is then normalized to obtain the normalized structural layer importance of each layer.

[0027] S3.3 Calculation of the importance of the fusion layer: Based on the normalized sensitivity and the normalized structural layer importance, the fusion layer importance of each layer is obtained by weighting and fusing according to a preset weight coefficient. The preset weight coefficient is used to balance the relative contributions of gradient sensitivity and structural importance.

[0028] S3.4 Determining the retention ratio: Set a global minimum retention ratio and a target average retention ratio, and perform interpolation mapping on the retention ratio within a preset range according to the importance of the fusion layer, so that the layers with higher fusion layer importance obtain a higher retention ratio, the layers with lower fusion layer importance are close to the global minimum retention ratio, and the average retention ratio of all layers is close to the target average retention ratio.

[0029] S3.5 Budget Constraints and Rank Discretization: Let the maximum rank achievable at each level be the corresponding maximum retained rank. Calculate the initial retained rank of each level according to the retention ratio and integerize the initial retained rank. In the presence of an overall parameter budget constraint, calibrate the retained rank of each level so that the sum of the retained ranks of each level meets the preset global budget requirement, and obtain the final hierarchical retained rank configuration for the singular value decomposition in step (4).

[0030] Furthermore, step (4) includes the following sub-steps:

[0031] S4.1 Singular Value Decomposition: Perform singular value decomposition on the weight matrix of each layer in the model to obtain the decomposition result consisting of the left singular vector matrix, the singular value diagonal matrix and the right singular vector matrix, and arrange the singular values ​​in descending order;

[0032] S4.2 Determination of Rank Retention and Selection of High-Rank Direction: Based on the rank retention assigned to the current layer in step (3), select the corresponding number of maximum singular values ​​and their associated singular vectors, and use the linear space spanned by the singular vectors as the high-rank subspace of the layer.

[0033] S4.3 Construction of low-rank subspace: The linear space spanned by the singular vectors other than the singular vectors corresponding to the high-rank subspace is used as the low-rank subspace of this layer; wherein the high-rank subspace and the low-rank subspace are orthogonal to each other, and are used to realize the explicit isolation of gradients in the knowledge preservation direction and the updatable direction in step (5).

[0034] Furthermore, step (5) includes the following sub-steps:

[0035] S5.1 Loss gradient calculation: During the training phase, the gradient tensor of the loss function with respect to the weight matrix of each layer is calculated for the current batch of training data, and used as the input for subsequent projection and update operations;

[0036] S5.2, High-rank subspace component removal: Based on the high-rank subspace constructed in step (4), construct the corresponding high-rank projection operator for each layer; remove the components of the gradient tensor in the high-rank subspace to obtain the residual gradient orthogonal to the high-rank subspace;

[0037] S5.3, Low-rank subspace projection and update direction determination: Project the residual gradient to the corresponding low-rank subspace to obtain the update gradient restricted within the low-rank subspace, and use the update gradient as the update direction of the weight matrix of this layer, and perform parameter updates only within the low-rank subspace;

[0038] S5.4, Layered learning rate adjustment: Based on the layered fusion importance obtained in step (3), the basic learning rate is scaled by layer to obtain the effective learning rate of each layer, wherein the effective learning rate of the layer with higher fusion importance is less than the effective learning rate of the layer with lower fusion importance.

[0039] S5.5 Sentinel Consistency Check and Rollback: During training, the sentinel samples are periodically input into the updated model, and the current model output is compared with the baseline output recorded in step (1); when the deviation exceeds a preset threshold, at least one of the following operations is performed:

[0040] (a) Revert the model weights to the state of the previous training round;

[0041] (b) Decrease the learning rate or increase the regularization strength;

[0042] (c) Increase the rank retention of the corresponding layer to expand the range of the high-rank subspace.

[0043] Secondly, this invention provides a system for preventing tampering with the core knowledge of a large-scale insurance contract interpretation model, including:

[0044] The sentinel sample construction and baseline output acquisition unit is used to construct a set of sentinel samples based on core knowledge such as the interpretation of insurance terms, standard Q&A, claims process, and financial principles, and to record the input, standard output, and evaluation method for each sentinel sample; and to input the sentinel sample into the base model to perform forward inference and obtain the corresponding baseline output.

[0045] The hierarchical sensitivity calculation unit is used to input the sentinel samples sequentially into the base model and construct a mask scalar target that focuses only on key output positions; it is used to perform backpropagation on the mask scalar target to obtain the gradient tensor of the weight matrix of each layer, and calculate the sensitivity index of each layer based on the norm of the gradient tensor; and it is used to aggregate the sensitivity indices corresponding to all sentinel samples to obtain the average sensitivity sequence arranged by layer.

[0046] The layer importance fusion and rank retention allocation unit is used to normalize the average sensitivity sequence; calculate the structural layer importance based on the similarity between the input and output features of each layer obtained from one forward propagation and perform normalization; perform weighted fusion of the normalized sensitivity and normalized structural layer importance according to preset weights to obtain the fusion importance of each layer; and determine the retention ratio of each layer according to the fusion importance under the constraints of the global minimum retention ratio and the target average retention ratio, and calculate the retention rank of each layer accordingly.

[0047] The subspace construction unit is used to perform singular value decomposition on the weight matrix of each layer to obtain singular values ​​and corresponding singular vectors; it is used to span the singular vectors corresponding to the first few singular values ​​into a high-rank subspace and the remaining singular vectors into a low-rank subspace according to the rank retention of the corresponding layer, so that the high-rank subspace and the low-rank subspace are mutually orthogonal.

[0048] The gradient double projection and robust update unit is used to calculate the gradient tensor of the loss function with respect to the weight matrices of each layer during the training phase; to remove the components of the gradient tensor in the high-rank subspace to obtain the residual gradient; to project the residual gradient to the corresponding low-rank subspace and update the weight matrices of each layer based on the projection result; and to periodically input the sentinel samples into the updated model, compare the current output with the baseline output, and perform backoff, learning rate adjustment and / or rank preservation adjustment when the deviation exceeds a preset threshold.

[0049] Beneficial Effects: This invention proposes a robust update method for large models based on sensitivity analysis and orthogonal subspace decomposition, which, compared with existing fine-tuning methods, has at least the following technical advantages:

[0050] (1) Achieve controllable maintenance and consistency monitoring of core insurance knowledge. By constructing a sentinel sample set and recording baseline output, periodic consistency checks can be performed on the outputs related to core insurance knowledge, such as clause interpretation, standard Q&A, claims process, and financial statements, during model training or continuous updates. When the output deviation exceeds the threshold, correction measures such as rollback or parameter adjustment can be triggered, thereby reducing the risk of unwanted drift in the expression of core knowledge during the update process and improving output consistency and reliability.

[0051] (2) Achieve quantitative assessment and adaptive resource allocation of hierarchical protection strength. By jointly assessing gradient sensitivity and structural similarity, the fusion importance of each layer is obtained, and the retention ratio / retention rank of each layer is determined under global budget constraints, so that parameter protection resources can be allocated more reasonably among different network layers. Compared with methods such as freezing parameters or unified regularization, this hierarchical allocation mechanism is conducive to reducing excessive disturbance to key layers and improving the structural stability of the update process.

[0052] (3) Implement subspace partitioning for knowledge preservation direction and task adaptation direction. High-rank subspace and low-rank subspace are constructed using singular value decomposition, and the protected direction and updatable direction in parameter space are partitioned using their orthogonality, reducing the possibility of mutual coupling of update directions in parameter space, thereby improving the controllability and numerical stability of model update process;

[0053] (4) Achieve robust updates and hierarchical adjustment based on dual gradient projection. By removing the gradient components in the high-rank subspace and projecting the residuals to the low-rank subspace, parameter updates are restricted to the updatable subspace. Combined with the hierarchical learning rate and consistency verification mechanism based on fusion importance, the update amplitude of different layers can be further controlled. While maintaining the stable expression of insurance core knowledge, it supports adaptation to new tasks, thereby achieving a more controllable trade-off between stability and adaptability. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of the overall process of the method of the present invention;

[0055] Figure 2 This is a flowchart of the steps of the method of the present invention;

[0056] Figure 3 This is a flowchart illustrating the sensitivity calculation in step S2 of the present invention.

[0057] Figure 4 This is a flowchart illustrating the specific steps of S3 in this method;

[0058] Figure 5 This is a flowchart showing the specific steps S4 and S5 of this method. Specific Implementation

[0059] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0060] As shown in the figure, this invention provides a method for preventing tampering with the core knowledge of a large-scale insurance contract interpretation model, including the following steps:

[0061] Step S1: Sentinel Sample Construction and Management

[0062] This step is one of the basic components of the method of the present invention, and is used to construct a sentinel sample set for monitoring the retention of core insurance knowledge.

[0063] In one embodiment of the present invention, the sentinel sample set is constructed based on core knowledge of the insurance industry, including but not limited to policy interpretations, standard Q&As, claims processes, and financial standards. For each sentinel sample, its input, the baseline output of the base model, and the corresponding output determination method (including evaluation rules and threshold parameters) are recorded. The sentinel sample set is used in subsequent model updates to periodically verify the consistency between the model output and the baseline output, and serves as the input benchmark for sensitivity calculation.

[0064] Step S2: Sensitivity calculation based on Jacobi

[0065] This step is one of the fundamental components of the method of this invention, used to evaluate the sensitivity of the weights of each layer of the model to the output of sentinel samples. In a preferred embodiment of this invention, sentinel samples are input into the base model and forward propagation is performed to obtain intermediate layer features, and a mask scalar target is constructed at the target output position; backpropagation is performed on the mask scalar target to obtain the gradient tensor of the target output with respect to the weights of each layer, wherein the gradient tensor can be regarded as an equivalent representation of the corresponding Jacobian information. Based on the norm of the gradient tensor or other indicators representing the gradient magnitude, a sensitivity metric for each layer is constructed and aggregated along the sentinel sample dimension to obtain layer-by-layer sensitivity results, which serve as the basis for subsequent layer importance fusion and parameter resource allocation. Through the above sensitivity calculation, network layers that are more sensitive to changes in sentinel sample output can be identified, providing a basis for subsequent layer retention strategies and update constraints.

[0066] Step S3: Layer Importance Fusion and Quota Allocation

[0067] This step is one of the fundamental components of the method of this invention. Based on the hierarchical sensitivity results obtained in step S2, it comprehensively determines the parameter retention strength and resource allocation for each network layer. In a preferred embodiment of this invention, based on the hierarchical sensitivity index obtained from sentinel samples, an index reflecting the structural similarity between input and output features is introduced as supplementary information. The sensitivity index and the structural similarity index are normalized respectively, and a unified importance index for each layer is obtained through linear weighting and / or other weighted fusion methods. Under the constraints of a preset global minimum retention ratio and a target average retention ratio, a corresponding retention ratio or retention rank is allocated to each layer according to the unified importance index, thereby forming a hierarchical parameter protection quota. This quota is used to balance maintaining the stable expression of core insurance knowledge with adapting to new task updates.

[0068] Step S4: Subspace Construction Based on SVD

[0069] This step is one of the fundamental components of the method of this invention. Based on the hierarchical retention quota determined in step S3, it divides the parameter space corresponding to the weight matrix of each layer into mutually orthogonal high-rank subspaces and low-rank subspaces. In one embodiment of this invention, singular value decomposition is performed on the weight matrix of the l-th layer to obtain singular vector directions sorted by singular value magnitude. According to step S3... Rank retention of layer assignment Before selection The singular vectors span the high-rank subspace of this layer; the remaining singular vectors span the low-rank subspace of this layer. Since the set of singular vectors obtained by singular value decomposition satisfies orthogonality, the high-rank subspace and the low-rank subspace are mutually orthogonal, thereby realizing the partitioning of the protected direction and the updatable direction in the parameter space, providing a subspace basis for subsequent constraint updates based on projection.

[0070] Step S5, Gradient Dual Projection and Robust Update

[0071] This step is one of the basic components of the method of the present invention. It is used to constrain the gradient update direction during the model training process, so that the parameter update is limited to the low-rank subspace constructed in step S4, thereby reducing the disturbance of the high-rank direction carrying the core knowledge of insurance during the update process.

[0072] In a preferred embodiment of the present invention, the first [item] obtained during the training process... Layer gradient The following two types of projection operations are performed sequentially. First, construct the... Projection matrix of the height-rank subspace The components of the gradient in the high-rank subspace are then removed to obtain the residual gradient. ,in Let be the identity matrix. Next, construct the... Projection matrix of the low-rank subspace of the layer The residual gradient is projected onto the low-rank subspace to obtain the update gradient for parameter updating. Based on the updated gradient As the first The layer weight parameters are updated, thus restricting the update direction to the low-rank subspace. Furthermore, this step includes periodic consistency checks: within a preset training interval, sentinel samples are input into the current model, and the deviation between their output and the baseline output recorded in step S1 is compared; when the deviation exceeds a preset threshold, at least one adjustment operation is triggered, including learning rate adjustment, rolling back to the previous training round state, and / or early termination of training.

[0073] The above steps S1 to S5 constitute the basic flow of the method of this invention, used to combine the maintenance of core insurance knowledge with the model update process. For ease of understanding, the specific implementation section below will further elaborate on each step and provide examples. It should be understood that the aforementioned sub-steps are used to illustrate one implementation of this invention, and those skilled in the art can add, delete, or adjust them without departing from the spirit of this invention; all such additions, deletions, or adjustments should fall within the protection scope of this invention.

[0074] Step S1: Sentinel Sample Construction and Management

[0075] Step S1 is one of the fundamental steps of the method of the present invention, used to construct a sentinel sample set for monitoring the retention of core insurance knowledge, and to provide an input benchmark for subsequent sensitivity calculation and consistency verification. In a preferred embodiment of the present invention, step S1 can be further refined into the following sub-steps S1.1 to S1.5 to illustrate the construction and management process of the sentinel samples. It should be understood that sub-steps S1.1 to S1.5 are preferred embodiments, and those skilled in the art can add, delete, or adjust them according to actual application scenarios. Such additions, deletions, or adjustments should not be construed as limiting the scope of protection of the present invention. Further, the detailed process of step S1 includes the following sub-steps:

[0076] S1.1 Knowledge Analysis and Source Determination: Based on the knowledge content in insurance business that needs to remain stable over a long period and should not be disturbed by the training process, the knowledge sources of the sentinel samples are determined. The knowledge content includes, but is not limited to, clause interpretation knowledge, standard and normative knowledge, fixed business rules, procedural knowledge, and question-and-answer information with uniquely identifiable output, in order to clarify the semantic scope and expected output format of the sentinel samples;

[0077] S1.2 Sample Design and Standard Output Construction: Based on the aforementioned knowledge sources, design input representations that can trigger the model to generate stable outputs, and set corresponding standard outputs for each sentinel sample. The input can be natural language descriptions and / or structured input formats; the standard output can be fixed text, structured information, and / or a numerical range that can be determined according to rules. To enhance representation coverage, multiple semantically equivalent input representations can be set for the same knowledge point;

[0078] S1.3 Sample Standardization and Storage: The constructed sentinel samples are standardized to include at least the sample identifier, input content, standard output, knowledge category identifier, evaluation method, and related judgment parameters; the standardized sentinel sample set is managed by version and stored in a storage medium accessible in the training process so that it can be reused in different training tasks or training stages.

[0079] S1.4 Sample Screening and Evaluation Method Setting: Candidate sentinel samples are screened to meet preset requirements in terms of business criticality, semantic clarity, output uniqueness, and evaluability; and corresponding evaluation methods are set according to knowledge categories. The evaluation methods include, but are not limited to, exact matching, template matching, tolerance range judgment, structural consistency judgment, or semantic similarity judgment; at the same time, thresholds or rule parameters are set for the evaluation methods for consistency monitoring in the subsequent training process.

[0080] S1.5 Baseline Output Acquisition: Perform forward inference on the sentinel sample using the current pedestal model, and record the model output as the baseline output of the corresponding sentinel sample; optionally, record the inference configuration parameters and / or model version information for subsequent comparative analysis; during training, do not perform backpropagation on the sentinel sample.

[0081] Through the above step S1, a set of sentinel samples that can characterize the core knowledge that needs to remain stable in the insurance field in the long term can be constructed, and corresponding baseline outputs and evaluation rules can be formed, thereby providing a reference benchmark for subsequent sensitivity assessment, hierarchical resource allocation and consistency verification based on sentinel samples.

[0082] Step S2: Jacobi-based sensitivity calculation

[0083] In a preferred embodiment of the present invention, step S2 can be further refined into the following sub-steps S2.1 to S2.4 to illustrate the implementation process of sensitivity calculation. It should be understood that sub-steps S2.1 to S2.4 are preferred embodiments, and those skilled in the art can add, delete, or adjust them according to actual needs, without limiting the scope of protection of the present invention. Further, the detailed process of step S2 includes the following sub-steps:

[0084] S2.1 Forward Propagation and Feature Recording: Sentinel Samples Input the data sequentially into the model, perform a complete forward propagation, and record the model's terminal output. It also records the activation information of intermediate layers required for backpropagation for subsequent gradient calculation;

[0085] S2.2 Scalar Target Construction: Construct a mask vector corresponding to the expected output position of the sentinel sample. The key output positions are set to 1, while the remaining positions are set to 0. A scalar target is defined based on this mask vector. To extract the scalar quantization target corresponding to the key output components;

[0086] S2.3, Backpropagation and Hierarchical Gradient Calculation: For the scalar target Perform one backpropagation and calculate the weight parameters of each layer. The gradient is denoted as ,in, ,in Characterizing the first The effect of layer weight changes on the key output components selected by the mask, and as an equivalent representation of the Jacobian information of the target output with respect to the layer weights;

[0087] S2.4 Sensitivity Measurement and Sample Aggregation: Gradient Tensors for Each Layer Calculate its Frobenius norm to obtain the sensitivity index of this layer to the s-th sentinel sample. Then, the sensitivity of all N sentinel samples at the same layer is averaged and aggregated to obtain the layer-average sensitivity. This results in sensitivity vectors arranged according to network hierarchy. This is used for layer importance fusion and quota allocation in subsequent step S3.

[0088] Step S3: Layer Importance Fusion and Quota Allocation

[0089] In a preferred embodiment of the present invention, step S3 can be further refined into the following sub-steps S3.1 to S3.5 to illustrate the calculation of layer importance and quota allocation process. It should be understood that sub-steps S3.1 to S3.5 are preferred embodiments, and those skilled in the art can add, delete, or adjust them according to actual needs, without limiting the scope of protection of the present invention. Further, the detailed process of step S3 includes the following sub-steps:

[0090] S3.1 Sensitivity Normalization: To reduce the dimensional impact of differences in parameter scale, activation distribution, and gradient magnitude among different layers, the layer-average sensitivity obtained in step S2 is normalized. Perform linear normalization to obtain , ,in To avoid constants with a denominator of zero, normalization allows sensitivity values ​​from different layers to be compared on a uniform scale.

[0091] S3.2, Calculation and Normalization of Structural Layer Importance: The structural layer importance is used to characterize the degree to which each layer preserves the structure of the input information. In a preferred embodiment, the intermediate features obtained from one forward propagation are used as the representation of the input information. Let the first... The layer input features and output features are respectively , ,in The batch size is used. For each sample in the batch, the cosine similarity is calculated and averaged over the batch dimension to obtain the structural importance of that layer. To reduce the differences in the range of structural importance values ​​across different layers, the following measures were taken: After normalization, we get... ;

[0092] S3.3, Calculation of Importance of Fusion Layer: After obtaining the normalized sensitivity... With normalized structural importance Then, the two are weighted and fused to obtain the fusion importance. In a preferred embodiment, a linear weighting method is used: in These are weighting coefficients, which can be tuned using a validation set or preset. The fusion importance... Used to characterize the overall degree of sensitivity and structural preservation properties of each layer;

[0093] S3.4 Determination of Retention Ratio: Let the global minimum retention ratio be... The target average retention rate is In one implementation, based on fusion importance... right Perform a linear mapping on the interval to obtain the first... Initial retention ratio of the layer ,in ;

[0094] S3.5 Budget Constraints and Rank Discretization: Let the first... The maximum rank that can be preserved by the layer is Based on the retention ratio obtained in step S3.4, calculate the retention rank of this layer. ,in This indicates rounding up. It is used when there are overall parameter budget constraints. At that time, it is possible to set Perform budget calibration to meet This yields the hierarchical rank-preserving configuration for subsequent step S4.

[0095] Through the above step S3, based on the simultaneous consideration of gradient sensitivity and structural importance, the retention ratio of each layer can be determined and discretized into a rank-retaining structure, providing input for the subsequent subspace construction based on singular value decomposition.

[0096] Step S4: Subspace Construction Based on SVD

[0097] In a preferred embodiment of the present invention, step S4 can be further refined into the following sub-steps S4.1 to S4.3 to more clearly illustrate the subspace construction process based on singular value decomposition. It should be understood that sub-steps S4.1 to S4.3 are preferred embodiments, and those skilled in the art can add, delete, or adjust them according to actual needs, without limiting the scope of protection of the present invention. Further, the detailed process of step S4 includes the following sub-steps:

[0098] S4.1 Singular Value Decomposition of Layer Weight Matrix: For the first layer weight matrix... Layer weight matrix Perform singular value decomposition to obtain in and These are column orthogonal matrices. It is a diagonal matrix of singular values, with the singular values ​​sorted in descending order. The decomposition result is used to provide the directions of the singular vectors sorted by singular values, in order to perform subsequent subspace partitioning;

[0099] S4.2 Determining the retained dimension: Calculate the first... Maximum rank achievable by a layer And based on the hierarchical retention ratio obtained in step S3 The number of singular values ​​to be retained in this layer (i.e., the rank to be retained) is determined as follows: The Used to determine the dimension of the high-rank subspace of this layer, the directions of the remaining singular vectors are used to construct the low-rank subspace;

[0100] S4.3 Subspace Construction: (This is related to the previous...) The direction of the singular vector corresponding to the singular value is defined as the high-rank direction, and the direction is constructed accordingly. High-rank subspace of the layer In one implementation, the high-rank subspace can be represented by the corresponding set of left singular vectors. Composed of, and formed by the corresponding set of right singular vectors The linear subspace representation is composed of, or determined by, both singular values. The singular vector directions corresponding to the remaining singular values ​​are constructed as a low-rank subspace. Based on the orthogonality of the set of singular vectors obtained from singular value decomposition, the high-rank subspace... With the low-rank subspace They are mutually orthogonal, thus completing the process for the first... Subspace partitioning of the layer parameter space.

[0101] Through step S4 above, based on the hierarchical rank-preserving configuration given in step S3, the high-rank subspace and low-rank subspace corresponding to each layer can be obtained, providing a subspace basis for gradient projection and parameter update in the subsequent step S5.

[0102] Step S5: Gradient Double Projection and Robust Update

[0103] In a preferred embodiment of the present invention, step S5 can be further refined into the following sub-steps S5.1 to S5.5 to more clearly illustrate the gradient projection update process based on the subspace. It should be understood that sub-steps S5.1 to S5.5 are preferred embodiments, and those skilled in the art can add, delete, or adjust them according to actual needs, without limiting the scope of protection of the present invention. Further, the detailed process of step S5 includes the following sub-steps:

[0104] S5.1 Loss gradient calculation: For the first... Layer weight matrix Calculate the loss function about gradient , the gradient As input for subsequent projection and update operations;

[0105] S5.2 High-rank component removal: Using the high-rank subspace constructed in step S4, the corresponding projection operator is... Gradient Projecting onto a high-rank subspace yields And calculate the residual gradient The residual gradient is used to characterize the gradient result after removing the high-rank directional components;

[0106] S5.3, Low-rank subprojection update: update the residual gradient Project into the low-rank subspace constructed in step S4 In a preferred embodiment, the projection operator corresponding to the low-rank subspace is utilized. get and with the stated As the first The gradient for updating layer weight parameters is used to perform parameter updates under low-rank subspace constraints.

[0107] S5.4, Layered Learning Rate and Regularization: To reflect the differences between different layers, the base learning rate can be scaled in layers. In a preferred embodiment, the fusion importance obtained in step S3 is used... Set the first The effective learning rate of the layer is ,in The learning rate is used as the base learning rate. In addition to learning rate scaling, the update magnitude can be controlled by adjusting the regularization coefficients, among other methods. The scaling method can be replaced or adjusted according to task requirements.

[0108] S5.5 Sentinel Consistency Check and Rollback: To monitor the retention of core knowledge during training, within a preset training interval, sentinel samples are input into the current model, and the difference between their output and the baseline output recorded in step S1 is compared to calculate the consistency deviation Δ. When Δ exceeds a preset threshold δ (δ can be determined through experience or a validation set), at least one correction measure is triggered, including: (a) rolling back to the previous round of training weights and canceling the current update; (b) reducing the effective learning rate of the corresponding layer. (c) Temporarily increase the rank retention factor and / or increase the regularization term coefficient; To adjust the high-rank subspace The coverage area.

[0109] Through the above step S5, the gradient can be processed by subspace projection during training, and the update process can be controlled by the consistency verification mechanism, thereby providing an implementation path for maintaining the core insurance knowledge and adapting the model for updates.

[0110] Example

[0111] In this embodiment, a 12-layer transformer architecture GPT-2 language model is used, with a hidden layer dimension of 768 and 12 attention heads. Using an insurance business scenario as a background, the requirements for stable representation of core insurance knowledge during continuous model updates are explained. This embodiment executes steps S1 to S5 sequentially, including sentinel sample construction, sensitivity calculation, layer importance determination, subspace construction, and gradient constraint update, to demonstrate an implementation of the method of this invention in the model training / update process.

[0112] Step 1 (Sentinel Sample Construction and Management)

[0113] In this embodiment, in order to monitor the model's retention of key knowledge points in insurance business, a sentinel sample set is first constructed based on insurance terms and common Q&A content. This sample set is then used for comparative evaluation before and after the model update to determine whether the model's output on core knowledge points has shifted.

[0114] 1.1 Knowledge Gathering and Source Determination: This embodiment selects several core knowledge points that need to remain stable over the long term from businesses such as auto insurance, accident insurance, and health insurance. For example, in traffic accident scenarios, when a third party is fully liable, does the insurance company have the right of subrogation? After screening, this embodiment determines approximately 20 core knowledge points as the knowledge sources for constructing the sentinel sample;

[0115] 1.2 Sample Design and Standard Output Construction: Based on the core knowledge points mentioned above, several input questions are designed for each knowledge point to cover different expressions. For example, for the knowledge point related to "subrogation right," 2-3 semantically equivalent but differently expressed input samples can be designed. Ultimately, this embodiment constructs approximately 50 sentinel samples and sets a corresponding unified standard output text for each sample for subsequent consistency judgment.

[0116] 1.3 Sample Standardization and Storage: After the sample construction is completed, all sentinel samples are recorded and managed in a structured form, including at least the following fields: sample number, knowledge point, input content, standard output, and evaluation method, so as to facilitate programmatic reading and reuse in subsequent processes;

[0117] 1.4 Sample Screening and Evaluation Method Setting: All candidate samples were manually reviewed, with a focus on eliminating samples with unclear expressions, potential model ambiguity, or whose business meaning had been updated. A small number of samples with semantic ambiguity were removed, ultimately retaining approximately 45 sentinel samples suitable for testing; and corresponding evaluation methods were set for these samples to monitor consistency during training or update processes.

[0118] 1.5 Baseline Output Acquisition: Input the sentinel samples into the currently used base model, and record the model output under the preset inference configuration as the baseline output of the corresponding samples. The baseline output is used as a reference benchmark in subsequent sensitivity calculation and consistency detection; during subsequent model updates, this sentinel sample set is only used for evaluation and monitoring and does not participate in training parameter updates.

[0119] Through step 1 above, this embodiment obtains a set of sentinel samples that cover the main core insurance knowledge points and can be managed in a structured manner, and forms a corresponding baseline output, providing a stable and reproducible input basis for subsequent sensitivity calculation and model update constraints based on sentinel samples.

[0120] Step 2 (Jacobi Sensitivity Calculation)

[0121] In this embodiment, in order to evaluate the influence of the parameters of each layer of the model on the output of the sentinel samples, the sentinel samples obtained in step 1 are sequentially input into the current model. Necessary intermediate results are recorded during the forward and backward propagation processes, and the sensitivity index of each layer is calculated accordingly, which serves as the input for subsequent layer importance fusion and retention quota allocation.

[0122] 2.1 Forward Propagation and Feature Recording: Using approximately 45 sentinel samples retained after filtering in Step 1, each sample is input into the current model and a complete forward inference is performed. During inference, the model generates intermediate activation features at each layer and obtains the final output vector at the output layer. This embodiment records the final output of each sample and saves the intermediate activation information required for backpropagation. To facilitate subsequent determination of the focus output position, this embodiment also records the corresponding focus output dimension index for samples of different business categories. For example, in samples related to "subrogation compensation," a dimension in the output vector corresponding to the business semantics is taken as the focus position, and the output value of that dimension and its corresponding intermediate layer activation features are recorded.

[0123] 2.2 Scalar Target Construction: After completing forward inference, the focus output position of the sample is determined according to its business category, and a scalar target for backpropagation is constructed accordingly. In one implementation, the output value corresponding to the focus output dimension is directly taken as the scalar target. For example, in a specific calculation, the focus output dimension corresponding to the "subrogation recovery" sample is the 105th dimension, so the output value of this dimension is used as the scalar target for the sample for subsequent backpropagation; the same applies to other knowledge point samples, where the output value of their corresponding focus output position is used as the scalar target.

[0124] 2.3 Backpropagation and Layered Gradient Calculation: Using the scalar objective constructed in S2.2 as the starting point for backpropagation, a complete backpropagation calculation is performed on the model to obtain the gradient results of each layer parameter relative to the scalar objective. This embodiment records and saves the gradients of all network layers for subsequent sensitivity statistics. For example, in a backpropagation of the "subrogation compensation" sample, it can be observed that the gradient magnitudes of layers 3, 7, and 11 are relatively larger; this embodiment saves the above gradient results along with the sample index as the basis for calculating the sensitivity of the sample in different layers. Other samples record their corresponding layered gradients according to the same process.

[0125] 2.4 Sensitivity Measurement and Sample Aggregation: After backpropagation of all samples, the gradient magnitudes of each sample at each layer are organized, and the sensitivity index of the sample at that layer is calculated. In one implementation, the sensitivities of multiple samples under the same knowledge point at the same layer are averaged to obtain the sensitivity of that knowledge point at that layer. For example, the knowledge point of "subrogation compensation" contains three samples. In this embodiment, the gradient magnitudes of the three samples at layers 3, 7, and 11 are extracted respectively, and averaged layer by layer to obtain the sensitivity of the knowledge point at the corresponding layer; other knowledge points are processed in the same way. Furthermore, the sensitivity results of all knowledge points at the same layer are summarized to form the hierarchical sensitivity results of the model, which are used for subsequent layer importance fusion calculations.

[0126] The process is as follows Figure 3 As shown.

[0127] Step 3 (Level Importance Integration and Quota Allocation)

[0128] In this embodiment, based on the hierarchical sensitivity results obtained in step 2, parameters for retaining proportions and retaining ranks are further assigned to each layer of the model. This step sequentially completes sensitivity normalization, structural index calculation and normalization, fusion importance generation, retention proportion mapping, and rank discretization and calibration under budget constraints.

[0129] 3.1 Sensitivity Normalization: In this embodiment, the average sensitivity of each layer obtained in step 2 is first normalized. Specifically, the maximum and minimum values ​​of the sensitivity of all layers are used as the endpoints of the interval, and the sensitivity of each layer is linearly mapped to... The interval is determined to eliminate the impact of differences in gradient magnitude dimensions between different layers. The normalized sensitivity result serves as one of the inputs for subsequent fusion calculations.

[0130] 3.2 Calculation and Normalization of Structural Layer Importance: During a forward inference process, this embodiment records the input and output features of each layer of the model and calculates the similarity between them as a structural index to characterize the degree to which the layer preserves the structure of the input information. Considering that the similarity values ​​of different layers may differ, this embodiment further linearly normalizes the structural index to... Interval. The normalized structural index and the normalized sensitivity of S3.1 are used together to calculate the fusion importance;

[0131] 3.3 Calculation of the Importance of the Fusion Layer: In this embodiment, the normalized sensitivity of S3.1 and the normalized structural index of S3.2 are weighted and fused according to preset weights to obtain the fusion importance of each layer. To highlight the need to retain features related to core business knowledge, the weight of the sensitivity item in this embodiment can be set to be relatively higher (the specific weight can be determined through validation sets or experience). The fusion importance is used to characterize the comprehensive degree of each layer in terms of "output sensitivity" and "structural preservation", and serves as the basis for subsequent allocation of retention ratios.

[0132] 3.4 Retention Ratio Determination: In this embodiment, a retention ratio is assigned to each layer based on the fusion importance. First, a global minimum retention ratio is set (e.g., 0.2), and then the fusion importance is set within... A linear mapping is performed within the interval to obtain the retention ratio of each layer, so that layers with higher fusion importance correspond to higher retention ratios, and layers with lower fusion importance are close to the minimum retention ratio. The retention ratio is used to subsequently calculate the retention rank of each layer.

[0133] 3.5 Budget Constraints and Rank Discretization: Based on the size of the weight matrix of each layer, this embodiment first calculates the maximum rank that can be retained in each layer, and then multiplies the retention ratio obtained in S3.4 by the maximum rank to obtain the initial retention rank. The initial retention rank needs to be integerized; this embodiment can use rounding down or rounding up to obtain the discrete retention rank. Further, the discrete retention ranks of all layers are summed and compared with the preset total budget; when the total budget is exceeded, this embodiment performs budget calibration on the retention ranks of each layer (e.g., proportional reduction or priority adjustment according to importance) until the budget constraint is met, thereby obtaining the final retention rank configuration of each layer, which is used for subsequent subspace construction and update constraints.

[0134] The process is as follows Figure 4 As shown

[0135] Step 4 (Singular Value Decomposition and Parameter Update)

[0136] In this embodiment, after completing the hierarchical rank-preserving configuration obtained in step 3, singular value decomposition is performed on the weight matrices of each layer of the model, and high-rank and low-rank subspaces for updating constraints are constructed based on the rank-preserving values ​​of each layer. This step includes singular value decomposition, determination of the rank-preserving dimension, and subspace construction.

[0137] 4.1 Singular Value Decomposition of Weight Matrix: In this embodiment, singular value decomposition is performed on the weight matrix of each layer of the model to obtain singular values ​​and their corresponding left and right singular vectors. The singular values ​​are sorted from largest to smallest and are used to characterize the main information distribution of the weight matrix in different directions. For example, for a certain layer with dimension... After decomposing the weight matrix, a set of singular values ​​and their corresponding left and right singular vectors are obtained in descending order. In this embodiment, the decomposition results are used for subsequent retention of direction selection and subspace construction, and the relevant vectors are recorded for future reference.

[0138] 4.2 Determining the Retained Dimension: In this embodiment, the retained rank of this layer is determined based on the rank obtained in step 3. (Or the rank of retention obtained by conversion from the retention ratio), select the top rank from the singular value sequence. The singular values ​​and their corresponding directions are selected as the primary directions to be retained. Optionally, if the maximum retainable rank of a certain layer is... And the retention ratio given in step 3 is Then the rank of this layer can be preserved by The result is obtained and converted to an integer; then the first integer is selected. The singular vectors corresponding to the singular values ​​are used to construct a high-rank subspace;

[0139] 4.3 Subspace Construction: After determining the singular values ​​to be retained and their directions, this embodiment will... The singular vector directions corresponding to the singular values ​​are combined to form the high-rank subspace of this layer, which is used to represent and carry the main expression directions related to the core insurance knowledge. Simultaneously, the singular vector directions corresponding to the remaining singular values ​​are combined to form the low-rank subspace, which is used to carry the updatable directions of the model under new tasks or new data. Based on the orthogonality of the singular vector set obtained from singular value decomposition, in one implementation, the high-rank subspace and the low-rank subspace are mutually orthogonal, thus achieving the partitioning of "protected directions" and "updatable directions" in the parameter space. For example, if the rank to be retained in a certain layer is determined to be... If the first 256 singular values ​​are selected, a high-rank subspace can be constructed in the direction corresponding to the first 256 singular values, and a low-rank subspace can be constructed in the remaining directions to form a complete subspace structure for this layer, which can be used for projection constraints in subsequent parameter update operations.

[0140] Step 5 (Gradient Dual Projection and Robust Update)

[0141] In this embodiment, after completing the subspace construction in step 4, the gradients generated during training are double-projected using the constructed high-rank and low-rank subspaces. This restricts parameter updates to the low-rank subspace, thereby reducing the disturbance to the high-rank direction carrying the core insurance knowledge during the update process. This step includes operations such as gradient calculation, high-rank component removal, low-rank direction update, and hierarchical learning rate and consistency verification.

[0142] 5.1 Loss Gradient Calculation: In this embodiment, during each training iteration, the gradient of the loss function with respect to the weights of each layer is first calculated, and the resulting complete gradient tensor is recorded as the input for subsequent projection operations. Taking the first... Taking a layer as an example, the shape of its gradient tensor is consistent with the weight matrix of that layer. In this embodiment, it is saved for subsequent high-rank / low-rank projection calculations.

[0143] 5.2 High-rank component removal: Using the high-rank subspace constructed in step 4, the gradient obtained in 5.1 is projected into the high-rank subspace to obtain the components of the gradient in the high-rank subspace; then, this component is removed from the original gradient to obtain the residual gradient. For example, if the dimension of the high-rank subspace in a certain layer is 256, this embodiment projects the gradient vector (or the representation of the gradient tensor under the corresponding subspace basis) in the 256 directions to obtain the high-rank components, and subtracts this component from the complete gradient to obtain the residual gradient that does not contain the high-rank direction components, which is used for subsequent low-rank direction updates;

[0144] 5.3 Low-rank direction update: In this embodiment, the residual gradient obtained in 5.2 is further projected onto the low-rank subspace to obtain low-rank gradient directions that can be used for updating. The parameter update is then performed using these low-rank gradients, thus restricting the update direction to the low-rank subspace. For example, when the dimension of the low-rank subspace of a certain layer is 128, this embodiment projects the residual gradient onto these 128 directions to form the final updatable gradient, and performs parameter updates for the weights of that layer accordingly.

[0145] 5.4 Layered Learning Rate and Regularization: To reflect the differences in importance among different layers, this embodiment applies different effective learning rates to each layer based on the fusion importance obtained in step 3: layers with lower importance can use relatively higher learning rates, while layers with higher importance can use relatively lower learning rates, in order to control the update magnitude of key layers. For example, in a training configuration, layers with lower importance can use 1.5 times the base learning rate, while layers with higher importance maintain the base learning rate or reduce it appropriately; in this embodiment, the low-rank gradients of each layer are scaled according to the corresponding effective learning rate during updates to complete the parameter updates. Optionally, the regularization coefficient can be adjusted to further constrain the update magnitude;

[0146] 5.5 Sentinel Consistency Verification and Rollback: To ensure the model maintains consistency in its representation of core insurance knowledge during training, this embodiment performs forward inference on all sentinel samples within a preset training interval and compares the current output with the baseline output recorded in step 1 to calculate the output deviation. When the deviation exceeds a preset threshold, a correction mechanism is triggered. For example, in a certain verification, if the output deviation corresponding to a certain knowledge point exceeds a preset threshold (e.g., 5%), this embodiment triggers at least one rollback or adjustment measure, including: restoring the weights to the previous round, reducing the learning rate of the corresponding layer, or increasing the rank retained by the layer to expand the coverage of the high-rank subspace, etc. The correction method can be selected according to the degree of deviation to control the stability of key knowledge directions.

[0147] like Figure 5 As shown.

[0148] Through steps S1 to S5 above, this embodiment fully demonstrates the application process of the method of the present invention in the actual model training process. The sentinel samples constructed in this embodiment are used to monitor changes in the model's output at key knowledge points; sensitivity calculation and layer importance allocation are used to determine the retention strength of different layers; and subspace construction and gradient dual projection achieve controlled updates in the low-rank direction. After consistency verification, the model can maintain a stable expression of the core insurance knowledge during the update process, verifying the effectiveness and applicability of the method of the present invention.

Claims

1. An insurance contract interpretation large model core knowledge tamper-proofing method, characterized in that, Includes the following steps: (1) Sentinel sample construction and baseline output acquisition: The sentinel sample set is constructed based on the core knowledge of the interpretation of insurance terms, standard Q&A, claims process and financial standards in insurance business; Record the input, standard output, and evaluation method for each sentinel sample; input the sentinel sample into the current base model to perform forward inference and obtain the corresponding baseline output, which is used for subsequent sensitivity calculation and consistency verification; (2) Jacobi-based hierarchical sensitivity calculation: The sentinel samples are sequentially input into the base model to construct binary mask / label targets corresponding to key output positions, and backpropagation is performed on the label targets to obtain the gradient tensor of the weight matrix of each layer; the sensitivity index of each layer is calculated based on the norm of the gradient tensor, and the sensitivity index of all sentinel samples is aggregated to obtain the average sensitivity sequence arranged by layer; (3) Layer importance fusion and rank retention allocation: The average sensitivity sequence is normalized; the structural importance of each layer is evaluated based on the similarity between the input features and output features of each layer; the normalized sensitivity and structural importance are fused according to preset weights to obtain the fused importance of each layer, and the corresponding retention ratio or retention dimension is assigned to each layer according to the fused importance, which is used to control the construction size of the high-rank subspace of each layer in the subsequent singular value decomposition. (4) Subspace construction based on singular value decomposition: Perform singular value decomposition on the weight matrix of each layer; according to the retention ratio or retention dimension obtained in step (3), select the corresponding number of singular values ​​and their singular vectors to form the high-rank subspace of the layer, and form the low-rank subspace with the remaining singular vectors, so that the high-rank subspace and the low-rank subspace are linearly orthogonal, thereby completing the subspace partitioning of the layer. (5) Gradient Dual Projection and Robust Update: After calculating the gradient of the loss function relative to the weight matrix of each layer in the subsequent training stage, the components of the gradient in the high-rank subspace direction are first removed, and then the remaining gradient is projected to the low-rank subspace through the projection operator to obtain the update gradient in the low-rank direction. Based on the update gradient, the corresponding weight parameters are updated only in the low-rank subspace direction, thereby reducing the disturbance of the knowledge representation direction in the high-rank subspace. During the training process, sentinel samples are periodically input into the model and compared with the baseline output. When the output deviates from the preset threshold, at least one of backoff, learning rate adjustment and retention rate adjustment is performed to maintain the stability of the model in the direction of insurance core knowledge.

2. The method of claim 1, wherein, Step (1) includes the following sub-steps: S1.1 Knowledge sorting and source determination: Based on the knowledge content in insurance business that needs to remain stable over a long period of time and should not be disturbed by the training process, the knowledge sources constituting the sentinel samples are determined. The knowledge content includes knowledge of clause interpretation, knowledge of standards and norms, fixed business rules, procedural knowledge, and question and answer information with unique output, which is used to clarify the semantic scope and expected output form of the sentinel samples. S1.2 Sample Design and Standard Output Construction: Based on the aforementioned knowledge sources, an input representation capable of triggering the model to generate stable output is designed for each knowledge point, and a corresponding standard output is set for each sentinel sample; the input is in the form of natural language description and / or structured input, and the standard output is fixed text, structured information, and / or a numerical range that can be determined according to rules; and multiple semantically equivalent input representations can be designed for the same knowledge point to enhance coverage; S1.3 Sample Normalization and Storage: The constructed sentinel samples are normalized to include at least sample identifier, input content, standard output, knowledge category identifier, evaluation method and evaluation threshold parameters. The normalized sample set is managed according to version and stored in a storage medium accessible to the training process so that it can be reused in different training tasks or training stages. S1.4 Sample Screening and Evaluation Method Setting: Candidate sentinel samples are screened to ensure that they meet the preset requirements in terms of business criticality, semantic clarity, output uniqueness and evaluability; and one or more evaluation methods are selected from exact matching, template matching, tolerance range judgment, structural consistency judgment and semantic similarity judgment according to different knowledge categories to monitor the consistency of model output. S1.5 Baseline Output Acquisition: Perform forward inference on the sentinel sample using the current pedestal model, and record the model output as the baseline output of the sentinel sample; and record the model version information and configuration parameters related to inference; do not perform backpropagation on the sentinel sample during subsequent training; the baseline output serves as a reference benchmark for sensitivity calculation and consistency verification, and is used to determine the retention status of the core insurance knowledge in the model during the update process.

3. The method of claim 1, wherein, Step (2) includes the following sub-steps: S2.1 Forward Propagation and Feature Recording: Input the sentinel samples into the base model to perform forward propagation, obtain the corresponding model output, and cache intermediate activation information at a preset layer to support subsequent gradient calculation; S2.2 Scalar Target Construction: For output positions that need to be kept stable, construct a binary mask vector aligned with the model output, and define a scalar target based on the binary mask vector to extract key output components, so that backpropagation only focuses on the key output positions; S2.3 Backpropagation and Layered Gradient Calculation: Perform backpropagation on the scalar objective to obtain the gradient tensor corresponding to the weight matrix of each layer; wherein, the gradient tensor is mathematically equivalent to outputting the weighted aggregation term of the Jacobian matrix with respect to the weights at the mask position, thus eliminating the need to explicitly construct the complete Jacobian matrix; S2.4 Sensitivity Measurement and Sample Aggregation: The Frobenius norm of the gradient tensor is used as the hierarchical sensitivity index of a single sentinel sample, and the hierarchical sensitivity index of all sentinel samples in the same layer is averaged to obtain the hierarchical average sensitivity, which is used for the layer importance fusion and quota allocation in step (3).

4. The method of claim 1, wherein, Step (3) includes the following sub-steps: S3.1 Sensitivity Normalization: In order to eliminate the influence of differences in parameter scale, activation distribution and gradient magnitude of different layers on the sensitivity measurement, the average sensitivity of the layers obtained in step (2) is linearly normalized to obtain the normalized sensitivity of each layer. S3.2 Calculation and Normalization of Structural Layer Importance: The intermediate features obtained from one forward propagation are used as input information; the cosine similarity between the input and output features of the l-th layer is calculated and averaged over the batch dimension to obtain the structural importance index of the layer; the structural importance index is normalized to obtain the normalized structural layer importance of each layer. S3.3 Calculation of the importance of the fusion layer: Based on the normalized sensitivity and the normalized structural layer importance, the fusion layer importance of each layer is obtained by weighting and fusing according to a preset weight coefficient. The preset weight coefficient is used to balance the relative contributions of gradient sensitivity and structural importance. S3.4 Retention Ratio Determination: Set a global minimum retention ratio and a target average retention ratio. Perform linear mapping processing on the retention ratio within a preset range according to the importance of the fusion layer, so that the layers with higher fusion layer importance obtain a higher retention ratio, the layers with lower fusion layer importance are close to the global minimum retention ratio, and the average retention ratio of all layers is close to the target average retention ratio. S3.5 Budget Constraints and Rank Discretization: Let the maximum rank that each layer can achieve be the corresponding maximum retained rank. Calculate the initial retained rank of each layer according to the retention ratio and integerize the initial retained rank. In the case of an overall parameter budget constraint, calibrate the retained rank of each layer so that the sum of the retained ranks of each layer meets the preset global budget requirement, and obtain the final hierarchical retained rank configuration for the singular value decomposition in step (4).

5. The method of claim 1, wherein, Step (4) includes the following sub-steps: S4.1 Singular Value Decomposition: Perform singular value decomposition on the weight matrix of each layer in the model to obtain the decomposition result consisting of the left singular vector matrix, the singular value diagonal matrix and the right singular vector matrix, and arrange the singular values ​​in descending order; S4.2 Determination of Rank Retention and Selection of High-Rank Direction: Based on the rank retention assigned to the current layer in step (3), select the corresponding number of maximum singular values ​​and their associated singular vectors, and use the linear space spanned by the singular vectors as the high-rank subspace of the layer. S4.3 Construction of low-rank subspace: The linear space spanned by the singular vectors other than the singular vectors corresponding to the high-rank subspace is used as the low-rank subspace of this layer; wherein the high-rank subspace and the low-rank subspace are orthogonal to each other, and are used to realize the explicit isolation of gradients in the knowledge preservation direction and the energy update direction in step (5).

6. The large model updating method of claim 1, wherein, Step (5) includes the following sub-steps: S5.1 Loss gradient calculation: During the training phase, the gradient tensor of the loss function with respect to the weight matrix of each layer is calculated for the current batch of training data, and used as the input for subsequent projection and update operations; S5.2, High-rank subspace component removal: Based on the high-rank subspace constructed in step (4), construct the corresponding high-rank projection operator for each layer; remove the components of the gradient tensor in the high-rank subspace to obtain the residual gradient orthogonal to the high-rank subspace; S5.3, Low-rank subspace projection and update direction determination: Project the residual gradient to the corresponding low-rank subspace to obtain the update gradient restricted within the low-rank subspace, and use the update gradient as the update direction of the weight matrix of this layer, and perform parameter updates only within the low-rank subspace; S5.4, Layered learning rate adjustment: Based on the layered fusion importance obtained in step (3), the basic learning rate is scaled by layer to obtain the effective learning rate of each layer, wherein the effective learning rate of the layer with higher fusion importance is less than the effective learning rate of the layer with lower fusion importance. S5.5 Sentinel Consistency Check and Rollback: During training, the sentinel samples are periodically input into the updated model, and the current model output is compared with the baseline output recorded in step (1); when the deviation exceeds a preset threshold, at least one of the following operations is performed: (a) Revert the model weights to the state of the previous training round; (b) Decrease the learning rate and / or increase the regularization strength; (c) Increase the retention rank of the corresponding layer to enhance the knowledge protection strength of the high-rank subspace.

7. A large model updating system for keeping insurance core knowledge, characterized by, include: The sentinel sample construction and baseline output acquisition unit is used to construct a set of sentinel samples based on core knowledge such as the interpretation of insurance terms, standard Q&A, claims process, and financial principles, and to record the input, standard output, and evaluation method for each sentinel sample; and to input the sentinel sample into the base model to perform forward inference and obtain the corresponding baseline output. The hierarchical sensitivity calculation unit is used to input the sentinel samples sequentially into the base model and construct a mask scalar target that focuses only on key output positions; it is used to perform backpropagation on the mask scalar target to obtain the gradient tensor of the weight matrix of each layer, and calculate the sensitivity index of each layer based on the norm of the gradient tensor; and it is used to aggregate the sensitivity indices corresponding to all sentinel samples to obtain the average sensitivity sequence arranged by layer. A layer importance fusion and rank-preserving allocation unit is used to normalize the average sensitivity sequence; This is used to calculate the structural layer importance based on the similarity between the input and output features of each layer obtained from a single forward propagation and to perform normalization processing; it is also used to perform weighted fusion of normalization sensitivity and normalized structural layer importance according to preset weights to obtain the fused importance of each layer; It is used to determine the retention ratio of each layer based on the fusion importance under the constraints of the global minimum retention ratio and the target average retention ratio, and to calculate the retention rank of each layer accordingly. The subspace construction unit is used to perform singular value decomposition on the weight matrix of each layer to obtain singular values ​​and corresponding singular vectors; it is used to span the singular vectors corresponding to the first few singular values ​​into a high-rank subspace and the remaining singular vectors into a low-rank subspace according to the rank retention of the corresponding layer, so that the high-rank subspace and the low-rank subspace are mutually orthogonal. The gradient double projection and robust update unit is used to calculate the gradient tensor of the loss function with respect to the weight matrix of each layer during the training phase; to remove the components of the gradient tensor in the high-rank subspace to obtain the residual gradient; and to project the residual gradient to the corresponding low-rank subspace and update the weight matrix of each layer based on the projection result. It is used to periodically input the sentinel samples into the updated model, compare the current output with the baseline output, and when the deviation exceeds a preset threshold, perform at least one of backoff, learning rate adjustment and rank retention adjustment.

Images