Semantic question and answer based model training method and device, equipment and storage medium

Abstract

The application discloses a model training method and device based on semantic question answering, equipment and a storage medium, relates to the technical field of data processing, and discloses a method comprising: constructing an adversarial sample corresponding to an original sample; after inputting question text information of the original sample and question text information of the adversarial sample into a semantic question answering model to be trained respectively, obtaining response data of each neural network level in the semantic question answering model to the original sample and response data of each neural network level in the semantic question answering model to the adversarial sample respectively; determining at least one fragile layer in the multiple neural network levels of the semantic question answering model based on the response data of the original sample and the response data of the adversarial sample; repairing a target fragile layer in the at least one fragile layer by using a pre-set repair strategy to obtain a repaired target question answering model. The application changes the model repair range from blind global parameter adjustment to accurate directional intervention, and improves the accuracy of the output answer of the semantic question answering model when facing the adversarial sample.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, device and storage medium for training models based on semantic question answering. Background Technology

[0002] With the widespread application of large-scale models in fields such as natural language processing and multimodal interaction, their security issues are becoming increasingly prominent. Large-scale language models refer to deep learning models trained on massive amounts of text data, enabling them to generate natural language text or understand the meaning of language text. Due to their complex parameter structures (typically reaching hundreds of billions) and open-ended generation capabilities, large-scale language models often face various adversarial attack threats. These attacks can induce large-scale language models to generate harmful content, leak sensitive information, or compromise system reliability, resulting in low security for large-scale language models.

[0003] Currently, to address the low security of large language models, existing defense solutions typically employ a "broad-based" strategy when training large language models for adversarial purposes. This involves adjusting the global parameters of the large language model to enhance its ability to cope with adversarial examples, thereby improving its security. However, the effect of global parameter adjustment on improving the security of large language models is limited, and their security remains low. When applied to scenarios such as semantic question answering, large language models produce low accuracy in their responses when facing adversarial examples. Summary of the Invention

[0004] The main purpose of this application is to provide a model training method, apparatus, device and storage medium based on semantic question answering, which aims to solve the technical problem that large language models have low accuracy in outputting answers when facing adversarial examples in related technologies.

[0005] To achieve the above objectives, this application proposes a model training method based on semantic question answering, the method comprising:

[0006] Construct adversarial samples corresponding to the original samples; wherein, the question text information of the adversarial samples is generated based on the question text information of the original samples under a semantic perturbation mechanism;

[0007] After inputting the question text information of the original sample and the question text information of the adversarial sample into the semantic question answering model to be trained, the response data of each neural network layer in the semantic question answering model to the original sample and the response data to the adversarial sample are obtained respectively.

[0008] Based on the response data of the original samples and the response data of the adversarial samples, at least one vulnerable layer in multiple neural network layers of the semantic question answering model is identified.

[0009] Using a pre-set repair strategy, the target vulnerable layer in at least one vulnerable layer is repaired to obtain the repaired target question-answering model.

[0010] In one embodiment, the response data includes the hidden state vector output by each neural network layer in the semantic question answering model after the question text information of the original sample and the question text information of the adversarial sample are respectively input into the semantic question answering model;

[0011] Based on the response data from the original samples and the response data from the adversarial samples, at least one vulnerable layer in multiple neural network layers of the semantic question answering model is identified, including:

[0012] For each neural network layer, the Euclidean distance between the hidden state vector corresponding to the original sample and the hidden state vector corresponding to the adversarial sample is calculated as the response offset difference; where the response offset difference characterizes the impact of the adversarial sample on the output of the neural network layer compared to the original sample.

[0013] From multiple neural network layers of the semantic question answering model, neural network layers with response offset differences greater than the first value are selected and placed into the target pool;

[0014] The neural network layers in the target pool are treated as at least one vulnerable layer.

[0015] In one embodiment, after calculating the Euclidean distance between the hidden state vector corresponding to the original sample and the hidden state vector corresponding to the adversarial sample as the response offset difference, the method further includes:

[0016] For each neural network layer, the mean of the response offset difference corresponding to the neural network layer is calculated after inputting the question text information of the original sample and the question text information of multiple adversarial samples into the semantic question answering model, and is used as the layer vulnerability index of the neural network layer.

[0017] From multiple neural network layers of the semantic question answering model, neural network layers with a vulnerability index greater than the second value are selected and placed into the target pool.

[0018] In one embodiment, after calculating the Euclidean distance between the hidden state vector corresponding to the original sample and the hidden state vector corresponding to the adversarial sample as the response offset difference, the method further includes:

[0019] Based on the response offset differences corresponding to multiple neural network layers, the propagation correlation coefficient matrix corresponding to the semantic question answering model is determined; where each element in the propagation correlation coefficient matrix represents the degree of correlation between response offsets between any two neural network layers.

[0020] From multiple elements of the propagation correlation coefficient matrix, select multiple target neural network layers whose element values ​​are greater than the third value;

[0021] Based on the hierarchical position of multiple target neural network layers in the semantic question answering model, the abnormal path for the semantic question answering model to respond abnormally to adversarial examples is determined.

[0022] In one embodiment, a pre-set repair strategy is used to repair a target vulnerable layer in at least one vulnerable layer to obtain a repaired target question-answering model, including:

[0023] Construct a target reward function based on at least one of the hierarchical vulnerability index of the target vulnerability layer, the output accuracy of the semantic question answering model, and the response latency of the semantic question answering model;

[0024] Based on the target reward function, a pre-set repair strategy is used to repair at least one target vulnerable layer in the vulnerable layer, resulting in a repaired target question answering model.

[0025] In one embodiment, based on a target reward function, a pre-set repair strategy is used to repair at least one target vulnerable layer in a vulnerable layer to obtain a repaired target question-answering model, including:

[0026] Based on the response offset difference of at least one vulnerable layer in descending order, a predetermined number of vulnerable layers are selected from at least one vulnerable layer as target vulnerable layers;

[0027] Based on the target reward function, the target vulnerable layer is repaired using a pre-set repair strategy to obtain the repaired first model;

[0028] Perform security and accuracy tests on the first model;

[0029] After determining that the first model passed both the security test and the accuracy test, the first model was selected as the target question-answering model after the repair.

[0030] In one embodiment, after performing security and accuracy tests on the first model, the method further includes:

[0031] After determining that the first model failed the security test or the accuracy test, the model parameters of the first model and the model parameters of the semantic question answering model are weighted and summed to obtain the model parameters of the repaired target question answering model.

[0032] In one embodiment, selecting a predetermined number of vulnerable layers from at least one vulnerable layer as target vulnerable layers includes:

[0033] Identify the sensitive layers in multiple neural network layers of a semantic question answering model;

[0034] From at least one vulnerable layer other than the sensitive layer, select a predetermined number of vulnerable layers as target vulnerable layers.

[0035] In one embodiment, constructing adversarial examples corresponding to the original samples includes:

[0036] Using a pre-set multi-dimensional perturbation strategy, semantic perturbation is performed on the question text information of the original sample to obtain at least one perturbated question text information.

[0037] Based on at least one perturbed question text, construct at least one candidate adversarial sample;

[0038] Based on the distribution of question text information in the original sample and at least one candidate adversarial sample, the output distribution difference value between the original sample and at least one candidate adversarial sample is determined; wherein, the output distribution difference value characterizes the degree of distribution difference of the output layer of the semantic question answering model after the question text information in the original sample and the candidate adversarial sample are respectively input into the semantic question answering model.

[0039] From at least one candidate adversarial sample, select candidate adversarial samples whose output distribution difference value is greater than the fourth value, and use them as adversarial samples.

[0040] In one embodiment, selecting candidate adversarial samples from at least one candidate adversarial sample whose output distribution difference value is greater than a fourth value as adversarial samples includes:

[0041] From at least one candidate adversarial sample, select candidate adversarial samples whose output distribution difference value is greater than the fourth value and whose cosine similarity is greater than the fifth value, and use them as adversarial samples.

[0042] Cosine similarity represents the semantic difference between the question text information of the original sample and the question text information of the candidate adversarial sample.

[0043] Furthermore, to achieve the above objectives, this application also proposes a model training device based on semantic question answering, the device comprising:

[0044] The construction module is used to construct adversarial examples corresponding to the original samples; the question text information of the adversarial examples is generated based on the question text information of the original samples under the semantic perturbation mechanism.

[0045] The acquisition module is used to acquire the response data of each neural network layer in the semantic question answering model to the original sample and the response data to the adversarial sample after inputting the question text information of the original sample and the question text information of the adversarial sample into the semantic question answering model to be trained, respectively.

[0046] A determination module is used to identify at least one vulnerable layer among multiple neural network layers of a semantic question answering model based on response data from raw samples and response data from adversarial samples.

[0047] The repair module is used to repair the target vulnerable layer in at least one vulnerable layer using a pre-set repair strategy, so as to obtain the repaired target question answering model.

[0048] In addition, to achieve the above objectives, this application also proposes a model training device based on semantic question answering. The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the model training method based on semantic question answering as described above.

[0049] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the semantic question-answering-based model training method described above.

[0050] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the semantic question-answering-based model training method described above.

[0051] One or more technical solutions proposed in this application have at least the following technical effects:

[0052] This application, based on a semantic question-answering scenario, first constructs adversarial samples corresponding to the original samples. Specifically, under a semantic-level perturbation mechanism, it generates adversarial sample question text information based on the question text information of the original samples. The question text information of the original samples and the adversarial samples are then input into the semantic question-answering model to be trained. The response data of each neural network layer in the semantic question-answering model to the original samples is used as a reference to compare the impact of the adversarial samples on the responses of each neural network layer in the semantic question-answering model. This allows for the identification of vulnerable layers in multiple neural network layers that are easily affected by adversarial samples. Specifically, it obtains the response data of each neural network layer in the semantic question-answering model to the original samples and the response data to the adversarial samples, and determines multiple neural network layers of the semantic question-answering model based on these response data. This paper describes a method for improving a semantic question answering model. It identifies at least one vulnerable layer within the model and then uses a repair strategy to repair the target vulnerable layer, resulting in a repaired target question answering model. Compared to related technologies that directly adjust global parameters to enhance the model's ability to cope with adversarial examples, which has limited effectiveness, this application compares the response data of each neural network layer in the semantic question answering model to original and adversarial examples. This effectively identifies vulnerable layers in the model susceptible to adversarial examples, shifting the repair focus from blind global parameter adjustments to precise targeted intervention. Based on effective model repair, this method minimizes the impact of the aforementioned security repair on the model's performance, effectively improving the model's security. Furthermore, when using the repaired model for semantic question answering, it effectively improves the accuracy of the model's output responses when facing adversarial examples. Attached Figure Description

[0053] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is one of the flowcharts illustrating the semantic question-answering-based model training method provided in this application;

[0056] Figure 2 This is the second flowchart illustrating the semantic question-answering-based model training method provided in this application;

[0057] Figure 3 This is the third flowchart illustrating the semantic question-answering-based model training method provided in this application;

[0058] Figure 4 This is the fourth flowchart illustrating the semantic question-answering-based model training method provided in this application;

[0059] Figure 5 This is the fifth flowchart illustrating the semantic question-answering-based model training method provided in this application;

[0060] Figure 6 This is the sixth flowchart illustrating the semantic question-answering-based model training method provided in this application;

[0061] Figure 7 This is the seventh flowchart illustrating the semantic question-answering-based model training method provided in this application;

[0062] Figure 8 This is a schematic diagram of the structure of the semantic question answering-based model training device provided in this application;

[0063] Figure 9 This is a schematic diagram of the structure of the semantic question-answering-based model training device provided in this application.

[0064] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0065] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0066] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit this application; the terms "comprising" and "having" and any variations thereof in the specification, claims and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0067] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0068] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0069] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0070] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0071] In related technologies, the main problems in large-scale adversarial evaluation and enhancement are as follows:

[0072] ① High cost of artificial red teams: Existing red team adversarial attack schemes require professional attackers to design adversarial samples, which is time-consuming and costly.

[0073] ② Vague security risk identification: Existing testing and evaluation schemes only report a decrease in accuracy or failure to meet security requirements, failing to identify vulnerable network layers and lacking a deep understanding of the network;

[0074] ③ Repairing damage to original performance: While optimizing the target large model to improve security, it will lead to a decrease in the accuracy of the original task.

[0075] Specifically, current adversarial attack techniques mainly rely on gradient perturbation or evolutionary algorithms to iteratively generate adversarial examples. Gradient perturbation, for example, employs the Fast Gradient Sign Method (FGSM), a classic method for generating adversarial examples. While these methods can expose surface vulnerabilities in a model, they have fundamental limitations.

[0076] Taking the widely used FGSM scheme as an example, the perturbations it generates are strictly limited to the range of tiny noise at the mathematical level, making it impossible to construct adversarial samples with semantic logic, such as injecting contradictory premises "if A then B, but not A still B", which makes it difficult to detect deep logical defects in text reasoning models.

[0077] Another type of genetic algorithm scheme can generate semantic perturbations, but it has huge computational overhead, takes a long time for a single attack, and can only report the attack success rate, but cannot locate the specific failed model components (such as abnormal attention mechanism).

[0078] Furthermore, existing defense strategies, such as adversarial training, employ a "wide net" approach, improving robustness through global parameter adjustments. However, this can easily lead to a significant decrease in the accuracy of the original mission, much like removing an entire organ to treat a local infection.

[0079] Furthermore, existing solutions, such as the genetic algorithm framework proposed by Alzantot et al., which deceives the language model through word substitution and sentence recombination, have the following problems, although they can achieve a high success rate of attack: First, the evolutionary process is completely decoupled from the model parameters, and the attack results can only reflect the superficial phenomenon of "the model being deceived", without revealing which layer of neurons has abnormal response. Developers need to manually analyze the fault points layer by layer. Second, the algorithm efficiency cannot meet the requirements of high-efficiency tasks. A single attack on the BERT-large model takes 62 minutes, resulting in 98% of long-tail scenarios (such as cross-cultural ambiguous expressions) not being fully validated.

[0080] To address the aforementioned issues, this application provides a model training method based on semantic question answering, aiming to shift the scope of repair in semantic question answering models from blind global parameter adjustment to precise targeted intervention, thereby solving the problem of ambiguous security risk localization. Furthermore, based on effectively repairing the model, the aforementioned security repair can minimize the impact on the model's performance, thus solving the problem of repair damaging the original performance. In addition, this application also provides a method for automatically constructing and screening adversarial examples that meet the requirements, solving the problem of high costs associated with manual red teams.

[0081] It should be noted that the executing entity of the embodiments of this application can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of realizing the above functions, a semantic question-answering-based model training device, etc. The embodiments of this application and the following embodiments will be described using a semantic question-answering-based model training device as an example.

[0082] The embodiments of this application and the following embodiments will be described in detail below.

[0083] This application provides a model training method based on semantic question answering, referring to... Figure 1 , Figure 1 This is one of the flowcharts illustrating the semantic question-answering-based model training method provided in this application. The method includes steps S101 to S104:

[0084] Step S101: Construct adversarial samples corresponding to the original samples;

[0085] Among them, the adversarial sample's question text information is generated based on the original sample's question text information under a semantic-level perturbation mechanism.

[0086] It should be noted that the original sample mentioned above can refer to natural input data (i.e., the question text information mentioned above) that has not been modified or disturbed in any way and can be correctly identified or processed by the model (such as a semantic question answering model). Adversarial samples can be input data generated by adding small perturbations to the original sample.

[0087] It should also be noted that, in the context of semantic question answering, the aforementioned semantic-level perturbation mechanism can be understood as a technique that generates adversarial examples by fine-tuning the semantic features of the input data without changing its core semantics.

[0088] Optionally, this embodiment generates adversarial sample problem text information through a semantic-level perturbation mechanism. For example, adversarial samples can be constructed by constructing multiple attack styles such as covering semantic logic, structural obfuscation, and contextual pollution.

[0089] For example, if the original sample's question text is: A is B's father, then the following adversarial sample's question text can be constructed:

[0090] ①The two have the same meaning but different ways of expressing it: A cannot possibly not be B's father; B is A's son, etc.

[0091] ② The two represent different logics (e.g., reconstructing a negation structure): A is not B's father, B is not A's son, etc.

[0092] Furthermore, contradictory premises can be implanted. For example, if the question text of the original sample is "If A then B", then the question text of the generated adversarial sample is "If not A, still B".

[0093] The above methods can be used to construct adversarial samples corresponding to the original samples, so that the original samples can be used as a reference to compare the impact of the adversarial samples on the semantic question answering model to be trained. The specific construction method is not limited in the embodiments of this application.

[0094] Step S102: After inputting the question text information of the original sample and the question text information of the adversarial sample into the semantic question answering model to be trained, the response data of each neural network layer in the semantic question answering model to the original sample and the response data to the adversarial sample are obtained respectively.

[0095] It should be noted that semantic question answering models typically include multiple neural network layers, specifically an input layer, multiple hidden / intermediate layers, and an output layer. These layers form a hierarchical collaborative relationship through data flow and parameter sharing, working together to complete the mapping from the original input to the predicted output.

[0096] For the original sample, after inputting the question text information of the original sample into the semantic question answering model to be trained, each neural network layer in the semantic question answering model will have response data. Similarly, after inputting the question text information of the adversarial sample into the semantic question answering model to be trained, each neural network layer in the semantic question answering model will also have response data. The above response data is obtained in this embodiment for subsequent comparison to determine the vulnerable layer in multiple neural network layers.

[0097] Step S103: Based on the response data of the original samples and the response data of the adversarial samples, identify at least one vulnerable layer among multiple neural network layers of the semantic question answering model.

[0098] Optionally, for each of the multiple neural network layers, the difference between the response data of the original sample and the response data of the adversarial sample can be compared separately, and some neural network layers can be identified as vulnerable layers based on the determined differences. For example, neural network layers with large differences can be identified as vulnerable layers, which are considered to be more susceptible to the influence of adversarial samples. These vulnerable layers can be repaired in the future to strengthen the model and improve its security.

[0099] Step S104: Using a pre-set repair strategy, repair the target vulnerable layer in at least one vulnerable layer to obtain the repaired target question-answering model.

[0100] Optionally, the above-mentioned repair strategies include, but are not limited to, at least one of the following: pruning strategy, weight reconstruction strategy and regularization enhancement strategy, and may also include other repair strategies, which are not limited in this application.

[0101] Among these strategies, the pruning strategy mentioned above includes, for example, pruning the l-th vulnerable layer by a ratio r; the weight reconstruction strategy mentioned above includes, for example, re-initializing the parameters of high-risk layers in the vulnerable layers using the Xavier method; and the regularization enhancement strategy mentioned above includes, for example, adding L2 regularization to the target vulnerable layer to suppress overactivation. The model repair strategies are shown in Table 1.

[0102] Table 1. Model Repair Strategies

[0103]

[0104]

[0105] It should also be noted that when repairing vulnerable layers in a semantic question-answering model, all identified vulnerable layers may be repaired, in which case all vulnerable layers are considered target vulnerable layers. Alternatively, only some target vulnerable layers among the identified vulnerable layers may be repaired to achieve a certain level of security. Or, the identified vulnerable layers may be repaired in batches, and the security of the entire model and the accuracy of the output answers may be tested and verified in a timely manner to achieve a certain level of security. This application does not impose any restrictions.

[0106] Specifically, in a semantic question-answering scenario, adversarial samples corresponding to the original samples can be constructed first. Under a semantic-level perturbation mechanism, adversarial sample question text information is generated based on the question text information of the original samples. The question text information of the original samples and the question text information of the adversarial samples are then input into the semantic question-answering model to be trained. The response data of each neural network layer in the semantic question-answering model to the original samples is used as a reference to compare the impact of the adversarial samples on the response of each neural network layer in the semantic question-answering model. This allows for the identification of vulnerable layers in multiple neural network layers that are susceptible to the influence of adversarial samples. Specifically, the response data of each neural network layer in the semantic question-answering model to the original samples and the response data to the adversarial samples are obtained. Based on these response data, at least one vulnerable layer in multiple neural network layers of the semantic question-answering model is identified. Then, a repair strategy is used to repair the target vulnerable layer in at least one vulnerable layer, resulting in a repaired target question-answering model.

[0107] This application provides a model training method based on semantic question answering. Compared with related technologies that directly improve the model's ability to cope with adversarial examples by adjusting global parameters, which has limited improvement effect, this application can effectively locate the vulnerable layers in the semantic question answering model that are susceptible to adversarial examples by comparing the response data of each neural network layer in the semantic question answering model to the original samples and adversarial samples. This shifts the scope of repair from blindly adjusting global parameters to precise targeted intervention. On the basis of effectively repairing the model, the above-mentioned security repair can minimize the impact on the model's performance, effectively improving the model's security. Furthermore, when using the repaired model for semantic question answering, the accuracy of the model's output answers when facing adversarial examples can be effectively improved.

[0108] In some embodiments, a specific implementation for determining at least one vulnerable layer is provided. Response data may include the hidden state vector output by each neural network layer in the semantic question answering model after the question text information of the original sample and the question text information of the adversarial sample are respectively input into the semantic question answering model;

[0109] Figure 2 This is the second flowchart illustrating the semantic question-answering-based model training method provided in this application, as shown below. Figure 2As shown, in Figure 1 Based on this, step S103 above includes:

[0110] Step S1031: For each neural network layer, calculate the Euclidean distance between the hidden state vector corresponding to the original sample and the hidden state vector corresponding to the adversarial sample, and use it as the response offset difference.

[0111] Among them, the response offset difference characterizes the impact of adversarial samples on the output of neural network layers compared to the original samples.

[0112] It should be noted that the aforementioned hidden state vector refers to the output vector of an intermediate layer in a multi-layered neural network, typically denoted as h. i , i represents the i-th intermediate layer.

[0113] For example, a layer-by-layer differential scanning method is first used to perform differential calculations on the hidden state vectors of the semantic question answering model under the original sample input and the adversarial sample input, in order to reveal the response shift of the anomalous input in the semantic question answering model, that is, the response shift difference δh. l :

[0114]

[0115] Where δ represents the Euclidean distance. and represents the hidden state h corresponding to the l-th neural network layer or neuron when adversarial samples and original samples are respectively input into the semantic question answering model. The larger the difference, the more sensitive the layer is to input perturbations.

[0116] Step S1032: From the multiple neural network layers of the semantic question answering model, select the neural network layers whose response offset difference is greater than the first value and put them into the target pool.

[0117] Step S1033: The neural network layers in the target pool are used as at least one vulnerable layer.

[0118] It should be noted that the first value mentioned above can be set according to the actual situation or through testing. For example, if the first value is set to 0.35, this value can be considered as the difference threshold. Neural network layers with response offset differences greater than the first value are placed in the target pool and are considered to be vulnerable layers that need to be focused on.

[0119] In this embodiment, Euclidean distance is specifically used to measure the difference in response offset between hidden state vectors in multiple neural network layers when facing original samples and adversarial samples. Based on this quantified difference and the difference threshold (i.e., the first value), the vulnerable layers in multiple neural network layers are accurately located, which facilitates subsequent accurate model repair. As a result, the output accuracy of semantic question answering is higher when using the repaired target question answering model.

[0120] In some embodiments, another specific implementation of identifying at least one vulnerable layer is provided. Following step S1031 above, the following steps may also be included:

[0121] Step S1-1: For each neural network layer, calculate the mean of the response offset difference corresponding to the neural network layer after inputting the question text information of the original sample and the question text information of multiple adversarial samples into the semantic question answering model, and use it as the layer vulnerability index of the neural network layer.

[0122] It should be noted that after inputting a set of "original samples-adversarial samples" into the semantic question answering model, the response offset difference corresponding to each neural network layer can be obtained. When multiple sets of "original samples-adversarial samples" are input into the semantic question answering model, the average of multiple response offset differences in each neural network layer can be taken as the layer vulnerability index of each neural network layer, so as to evaluate the vulnerability of each neural network layer in the face of a large number of adversarial samples.

[0123] Step S1-2: From the multiple neural network layers of the semantic question answering model, select neural network layers with a layer fragility index greater than the second value and put them into the target pool.

[0124] It should be noted that the second value mentioned above can be set according to the actual situation or through testing. This value can be considered as the threshold of the layer fragility index. When the layer fragility index is greater than the second value, the neural network layer can be considered as a fragile layer that needs to be focused on and can be put into the target pool for subsequent repair operations.

[0125] For example, the Vulnerability Index (Vulnerability Index) can be used. l Perform an aggregated risk assessment for each layer:

[0126]

[0127] Among them, ||·|| F Describing the Frobenius norm, Let N be the state difference of the i-th sample at the l-th neural network layer (represented by the response offset difference), and N be the total number of samples corresponding to the l-th neural network layer.

[0128] Optionally, the index mean Vul can be used as the basis for different samples. l A three-dimensional heatmap (Layer×Token×Channel) is constructed to graphically display the intensity of abnormal responses, forming a "vulnerable hotspot" map. For example, in a certain semantic question answering model, it is shown graphically that Layer-17, Layer-24, and Layer-31 often exhibit high hot responses when processing counterfactual reasoning adversarial examples, indicating that they are potential security risk layers that need to be repaired.

[0129] In some embodiments, a specific implementation is provided for determining the abnormal response path of a semantic question-answering model in response to adversarial examples. Following step S1031 above, the following steps may also be included:

[0130] Step S2-1: Based on the response offset differences corresponding to multiple neural network layers, determine the propagation correlation coefficient matrix corresponding to the semantic question answering model;

[0131] Each element in the propagation correlation coefficient matrix represents the degree of correlation between any two neural network layers that result in response shifts.

[0132] Step S2-2: From the multiple elements of the propagation correlation coefficient matrix, select the multiple target neural network layers whose element values ​​are greater than the third value.

[0133] It should be noted that the third value mentioned above can be set according to the actual situation or through testing. This value can be considered as a threshold for the degree of correlation of response offset between multiple neural network layers.

[0134] Step S2-3: Based on the hierarchical position of multiple target neural network layers in the semantic question answering model, determine the abnormal path in which the semantic question answering model responds abnormally to adversarial examples.

[0135] Specifically, to further identify the propagation paths of abnormal behavior caused by the vulnerable layer (i.e., the aforementioned abnormal paths), embodiments of this application construct a propagation correlation coefficient matrix based on cross-layer state changes:

[0136]

[0137] This matrix reveals the degree of anomalous activation coupling between any two hidden state layers. If a certain layer has a high positive correlation with multiple downstream layers, such as ρ>0.6, it can be determined that it has the characteristics of a "core lesion". The system can further construct a propagation map to track anomalous paths and record the vulnerability contribution on each path.

[0138] Optionally, to accurately pinpoint the source neural layer of abnormal behavior, the system performs column-by-column distortion analysis on the weight matrix of high-risk layers within the vulnerable layers. The cosine similarity of each column vector before and after perturbation is used as an indicator. If ω sim A value <0.6 indicates that the parameters of this neural network layer are severely affected and need to be included in the "root cause candidate set". In this embodiment, these neural layers are mapped back to the original task structure (such as attentionheads and FFN nodes) to achieve three-dimensional fusion localization of "function-structure-abnormality". This ultimately forms a "key neural layer / meta-index table" as direct input for subsequent repair strategies.

[0139] In some embodiments, a specific implementation is provided for repairing the target vulnerability layer in a semantic question answering model.

[0140] Figure 3 This is the third flowchart illustrating the semantic question-answering-based model training method provided in this application, as shown below. Figure 3 As shown, in Figure 1 Based on this, step S104 above includes:

[0141] Step S1041: Construct a target reward function based on at least one of the hierarchical vulnerability index of the target vulnerability layer, the output accuracy of the semantic question answering model, and the response latency rate of the semantic question answering model.

[0142] It should be noted that the output accuracy of the above semantic question answering model is the accuracy of the output answer of the semantic question answering model. The larger the value, the better the model performance. The response latency of the above semantic question answering model is the delay time from the input question text information to the output answer text information. The smaller the value, the better the model performance.

[0143] Optionally, to guide the strategy towards improving security while maintaining performance, the system is designed with the following dual-objective reward function R. t :

[0144] R t =0.7×(-Δ∑Vul) l )+0.25×ΔAcc+0.05×(-ΔLatency);

[0145] Among them, the first term (i.e., the hierarchy vulnerability index Vul) lThe first term encourages reducing security risks, the second term (i.e., the output accuracy of the semantic question answering model) encourages improving accuracy, and the third term (i.e., response latency) penalizes increased response overhead. In this embodiment, the function feedback is used as the core signal for reinforcement learning optimization to train a PPO (Proximal Policy Optimization) network for long-term action policy optimization. PPO is a reinforcement learning algorithm designed for efficient and stable agent training.

[0146] Step S1042: Based on the target reward function, repair the target vulnerable layer in at least one vulnerable layer using a pre-set repair strategy to obtain the repaired target question answering model.

[0147] In some embodiments, a specific implementation is provided for repairing the target vulnerability layer to obtain a repaired target question-answering model. Figure 4 This is the fourth flowchart illustrating the semantic question-answering-based model training method provided in this application, as shown below. Figure 4 As shown, in Figure 3 Based on this, step S1042 above includes:

[0148] Step S1042-1: Select a preset number of vulnerable layers from at least one vulnerable layer in descending order of the response offset difference of at least one vulnerable layer as target vulnerable layers.

[0149] It should be noted that, in this embodiment of the application, the response offset difference is used as the key indicator of concern, and vulnerable layers (considered high-risk layers) with large response offset differences are prioritized for repair. However, other indicators can also be used as the focus, such as the layer vulnerability index Vul. l As a key indicator of focus, the Vulnerability Index (Vulnerability Index) will be prioritized. l This application does not impose any restrictions on repairing larger vulnerable layers (considered high-risk layers).

[0150] Optionally, a progressive repair protocol can be adopted, which intervenes in only a small number (1-2) of the target vulnerable layers each time, while prioritizing the repair of high-risk layers. After repair, the state is reassessed and the process proceeds to the next round, thus avoiding excessive impact of the repair process on the model's parameters and performance.

[0151] In some embodiments, selecting a predetermined number of vulnerable layers from at least one vulnerable layer as the target vulnerable layer may include:

[0152] Identify the sensitive layers in multiple neural network layers of a semantic question answering model;

[0153] From at least one vulnerable layer other than the sensitive layer, select a predetermined number of vulnerable layers as target vulnerable layers.

[0154] Specifically, to prevent damage to the core parameters of the model, the system can enable a "core layer protection mechanism" during the repair process, skipping operations on sensitive layers such as LayerNorm and Embedding. Through multiple iterations, it converges to a set of optimized weights, completing the final enhanced output of the model.

[0155] Step S1042-2: Based on the target reward function, the target vulnerable layer is repaired using a pre-set repair strategy to obtain the repaired first model.

[0156] Step S1042-3: Perform security and accuracy tests on the first model.

[0157] It should be noted that the above security tests are used to verify the model's ability to cope with adversarial examples. For example, they can be evaluated by the above response offset difference and / or hierarchical vulnerability index, which are not limited in this application.

[0158] It should also be noted that the accuracy test mentioned above is used to verify the accuracy of the model's output answer, and can be verified using model accuracy testing methods in related technologies.

[0159] Step S1042-4: After determining that the first model has passed the security test and the accuracy test, the first model is used as the target question-answering model after repair.

[0160] For example, if the repaired model meets the specified requirements, such as an overall security risk reduction of >60%, accuracy fluctuation of <±0.5%, and response latency increase of <8%, the model can be considered successfully repaired, and the current model parameters are frozen as the "enhanced model" output, which is the target question-answering model. If the requirements are not met, the previous state is reverted, and the "strategy inversion" channel is entered to modify the repair strategy. For example, other repair strategies or combinations of repair strategies are used to repair the repaired vulnerable layers, or to repair other unrepaired vulnerable layers, until the repaired model meets the specified requirements.

[0161] In some embodiments, another specific implementation is provided to repair the target vulnerability layer to obtain a repaired target question-answering model. After steps S1042-3 above, the following steps may also be performed:

[0162] After determining that the first model failed the security test or the accuracy test, the model parameters of the first model and the model parameters of the semantic question answering model are weighted and summed to obtain the model parameters of the repaired target question answering model.

[0163] Specifically, if the repaired first model fails the security test or the accuracy test, the model parameters of the first model can be weighted and summed with the model parameters of the original semantic question answering model, for example, using the following formula:

[0164]

[0165] in, These are considered to be the model parameters of the target question-answering model. It is considered to be the model parameters of the original semantic question answering model. The model parameters of the first model after repair are considered to be α, which is the weight. Generally, α is compensated starting from a very small value (such as 0.01) and gradually increased until the model performance accuracy (that is, the accuracy of the output answer) meets the standard.

[0166] It should be noted that the compensated large model still needs to undergo security testing to ensure that the compensated model does not introduce new security risks. After passing the security test, the target question-answering model can be obtained.

[0167] In some embodiments, a specific implementation of constructing adversarial examples corresponding to the original samples is provided. Figure 5 This is the fifth flowchart illustrating the semantic question-answering-based model training method provided in this application, as shown below. Figure 5 As shown, in Figure 1 Based on this, step S101 above includes:

[0168] Step S1011: Use a pre-set multi-dimensional perturbation strategy to semantically perturb the question text information of the original sample to obtain at least one perturbated question text information.

[0169] It should be noted that the aforementioned multi-dimensional perturbation strategies can be obtained from a pre-set multi-dimensional perturbation strategy library. That is, adversarial examples are updated through this library, which covers three types of attacks: logical obfuscation, semantic hijacking, and contextual pollution, achieving breakthroughs in multiple areas such as model attention mechanisms and inference chains. Table 2 below shows some examples from the multi-dimensional perturbation strategy library:

[0170] Table 2 shows some examples from the multidimensional perturbation strategy library.

[0171] Attack type Implementation paradigm Target security risks Logical confusion Insert contradictory premise: If A then B, but if not A, then B. The chain of reasoning breaks semantic hijacking Reconstructing the negation structure: non-rejection → rejection Conclusion Reversal Vulnerability Contextual pollution Inserting irrelevant clauses disrupts thematic consistency Attention mechanism failure

[0172] Step S1012: Based on at least one perturbed question text information, construct at least one candidate adversarial sample.

[0173] Step S1013: Based on the distribution of problem text information between the original sample and at least one candidate adversarial sample, determine the output distribution difference value between the original sample and at least one candidate adversarial sample;

[0174] The output distribution difference value represents the degree of distribution difference in the output layer of the semantic question answering model after the question text information in the original sample and the candidate adversarial sample are respectively input into the semantic question answering model.

[0175] Optionally, the above output distribution difference value can be characterized by the KL (Kullback-Leibler) divergence value, which is used to measure the degree of difference between the two probability distributions.

[0176] Step S1014: From at least one candidate adversarial sample, select candidate adversarial samples whose output distribution difference value is greater than the fourth value, and use them as adversarial samples.

[0177] It should be noted that the fourth value mentioned above can be set according to the actual situation or through testing. This value can be used as a threshold for the difference in output distribution. When the difference in output distribution of a candidate adversarial sample is greater than the fourth value, it is considered that the candidate adversarial sample has a large difference from the original sample output. Using such candidate adversarial samples makes it easier to conduct adversarial training effectively, thereby improving the model training effect.

[0178] For example, firstly, by monitoring distribution shift, the difference in output distribution between the original sample and the adversarial sample is calculated using the KL divergence metric model:

[0179]

[0180] Among them, P adv and P clean Let D represent the Prompt (i.e., the question text information mentioned above) of the adversarial sample and the original sample, respectively. Let x represent the x-th sample in the adversarial sample and the original sample, and let X represent the total number of samples in the adversarial sample and the original sample. KL If the value does not exceed the set threshold (e.g., dialogue model < 2.5), the adversarial sample perturbation upgrade will be automatically initiated to make the output distribution of the generated adversarial sample significantly different from that of the original sample, thereby improving the quality of the adversarial sample.

[0181] In some embodiments, another specific implementation is provided for constructing adversarial examples corresponding to the original sample. Figure 6 This is the sixth flowchart illustrating the semantic question-answering-based model training method provided in this application, as shown below. Figure 6 As shown, in Figure 5 Based on this, step S1014 above can be replaced by step S1014-1:

[0182] Step S1014-1: From at least one candidate adversarial sample, select candidate adversarial samples whose output distribution difference value is greater than the fourth value and whose cosine similarity is greater than the fifth value, and use them as adversarial samples;

[0183] Cosine similarity represents the semantic difference between the question text information of the original sample and the question text information of the candidate adversarial sample.

[0184] It should be noted that the fifth value mentioned above can be set according to the actual situation or through testing. For example, it can be set to 0.3, which can be used as the threshold for cosine similarity. If the cosine similarity between the question text information of the candidate adversarial sample and the question text information of the original sample exceeds the fifth value, it is considered that the semantic difference between the question text information of the constructed candidate adversarial sample and the question text information of the original sample is too large, and there is a case of fabrication. These samples need to be excluded in order to improve the overall quality of adversarial samples.

[0185] For example, when updating adversarial examples, semantic legitimacy constraints are applied. The cosine similarity of the embedding vectors is used to determine the semantic consistency of the sentences before and after the perturbation (i.e., the semantics of the question text information of the adversarial example and the question text information of the original sample), to avoid large semantic differences caused by fabricated new adversarial example prompts. The cosine similarity is calculated using the following formula:

[0186] d sem =1-cos(Embed(P) t Embed(P) t+1 ));

[0187] Among them, Embed(P t ) represents the embedding vector calculated by embedding models such as gte-large. For example, d sem If the value is greater than 0.3, regularization correction can be automatically initiated to ensure that the generated "doctor recommends stopping medication but immediately increasing the dosage" type of attack is syntactically correct but logically conflicting and effective against injection attacks.

[0188] The following example illustrates the semantic question-answering-based model training method of this application.

[0189] The existing large-scale model security assessment technology has several problems, mainly in the following aspects: First, because the design of adversarial examples relies on human experience, professional red teams need to spend a lot of time constructing effective attack cases, making it difficult to fully cover long-tail scenarios such as language ambiguity and cultural differences; Second, the existing assessment only stays at the model output level and cannot penetrate the black box to locate the specific failed neuron modules or neural network layers, forcing developers to adopt a global parameter adjustment repair strategy; Finally, although this crude global repair can temporarily improve model security, it significantly impairs the original task processing capability due to indiscriminate compression of model capacity or modification of model weight parameters, forming a vicious cycle of "repair-performance degradation-re-repair".

[0190] To address the aforementioned issues, this application provides a model training method based on semantic question answering and proposes a solution for automated adversarial assessment and enhancement of large models: Through automated semantic perturbation generation technology, deep attack samples such as logical contradictions are directly constructed, significantly reducing reliance on manual red teams; combined with a neuron-level heatmap diagnostic mechanism, key parameter blocks causing decision-making biases in the model are identified, shifting the scope of remediation from blind global adjustments to precise targeted intervention; relying on a dual-objective optimization framework of security and performance, the core capabilities of the model are maintained while eliminating high-risk vulnerabilities. This closed-loop system not only solves the coverage blind spots in vulnerability detection but also overcomes the dilemma of remediating performance damage, making the security hardening of large models more accurate and sustainable.

[0191] Figure 7 This is the seventh flowchart illustrating the semantic question-answering-based model training method provided in this application, as shown below. Figure 7 As shown, the overall technical process is divided into six stages: automated adversarial example generation, security detection engine, vulnerability layer localization, targeted repair optimizer, security verification, and large model performance verification, constructing a closed-loop large model security assessment and enhancement system. The entire process uses the target large model (such as the semantic question-answering model to be trained in this application) as the core input, and through automatically constructing adversarial example-driven testing, refined neural network layer diagnosis, and reinforcement learning-based repair mechanisms, forms a highly efficient automated iterative mechanism of "attack-detection-localization-repair-verification." This provides a novel assessment system and security capability enhancement solution for large model security.

[0192] The following section provides a detailed technical process, using a large target model as an example of a semantic question-answering model:

[0193] S1. For semantic question answering models, this application first generates adversarial examples through a semantic-level perturbation mechanism, constructing adversarial examples covering multiple attack styles such as semantic logic, structural confusion, and contextual pollution. In this stage, distribution shift is used as the criterion. By calculating the distribution difference (such as KL divergence) between the adversarial examples and the original samples in the model output layer, the perturbation enhancement module is automatically triggered to ensure that the generated samples have logical consistency but inference ambiguity, thus achieving the effectiveness of semantic attacks.

[0194] S2, the generated adversarial examples are input into the security detection engine to test the model's security. The engine evaluates the model's response to the generated adversarial examples in real time, constructs a security score matrix, and simultaneously triggers the neural network layer differential analysis module to identify potential vulnerable hidden layers in the model when processing these anomalous inputs.

[0195] S3 achieves precise localization of vulnerable neural network layers and fault propagation paths through methods such as cross-layer neuron state difference analysis, heatmap visualization, propagation path analysis, and weight distortion detection. For example, Euclidean distance (δ) and the layer vulnerability index (Vul) are used as core indicators to assist in constructing a propagation correlation coefficient matrix (ρ), enabling quantitative reduction of the root cause of the fault and providing target guidance for subsequent remediation strategies.

[0196] S4. Based on the identified vulnerable layers and neural networks, this application introduces a repair decision-making mechanism based on a reinforcement learning strategy. The repair state vector is defined as follows:

[0197] s t = [Vul1, Vul2, ..., Vul L Acc test Latency] T ;

[0198] Among them, Vul l Acc represents the vulnerability index of layer l. test Let represent the accuracy of the current semantic question answering model on the validation set, and represent the response latency. Three optimization strategies are selected in the action space: high-risk neuron pruning, weight reinitialization, and L2 regularization. A dynamic balance between improving model safety and protecting large-scale model performance is achieved through the following bi-objective reward function:

[0199] R t =0.7×(-Δ∑Vul) l )+0.25×ΔAcc+0.05×(-ΔLatency).

[0200] S5. The repaired semantic question answering model undergoes another security test against adversarial examples. If it fails, it automatically returns to the sample generation stage (S1) for a new round of iterative detection. If it passes, it proceeds to the next stage (S6).

[0201] S6. After the safety standards are met, the system further verifies the model's accuracy and response efficiency. If the performance metric fluctuations are within the allowable range (e.g., accuracy decrease <0.5%), the model enhancement is considered successful, and the enhanced new large model (such as the target question-answering model mentioned above) is output. If the standards are not met, a compensation mechanism is activated. The compensation mechanism works by using the original weight parameters of the large model and the repaired parameters to perform a weighted summation to obtain the corresponding compensation parameters for the updated large model's weight parameters. The formula is as follows:

[0202]

[0203] Compensation typically starts with a very small value (e.g., 0.01) and gradually increases until the performance accuracy meets the standard. At this point, the compensated large model needs to return to step S1 to continue security testing to ensure that the compensated model does not introduce new security risks.

[0204] The following is a detailed introduction to the three innovative solutions proposed in this application in steps S1 to S4 above: semantic-level adversarial example automatic generation technology, neural network-level vulnerability localization technology, and dual-target dynamic repair technology:

[0205] ① Semantic-level adversarial example automatic generation technology:

[0206] This application introduces the generation concept of "semantic consistency and logical ambiguity" based on traditional numerical perturbation methods. The scheme includes the following key components:

[0207] First, the difference in output distribution between the original sample and the adversarial sample is measured by the KL divergence model through distribution offset monitoring. The calculation formula is detailed in the above embodiment and will not be repeated here.

[0208] When the output distribution difference value does not exceed the set threshold (e.g., dialogue model < 2.5), the adversarial sample perturbation upgrade will be automatically initiated. Specifically, the adversarial samples can be updated through a multi-dimensional perturbation strategy library. This strategy library covers three types of attack methods: logical confusion, semantic hijacking, and contextual pollution. It can achieve breakthroughs in multiple aspects such as model attention mechanism and inference chain.

[0209] Furthermore, semantic legitimacy constraints can be applied when updating adversarial examples. The semantic consistency of sentences before and after perturbation can be determined by the cosine similarity of the embedded vectors. This avoids large semantic differences caused by fabricated new adversarial examples, ensuring that the generated adversarial injections, such as "the doctor recommends stopping the medication but immediately increasing the dosage," are grammatically correct but logically conflicting.

[0210] ② Neural network-level vulnerability localization technology:

[0211] Based on existing methods for interpretability of neural networks, this application constructs a composite vulnerability localization mechanism of "multidimensional difference – thermal identification – path tracing – root cause reduction" to accurately track the performance and propagation path of abnormal model behavior caused by adversarial sample injection in the network structure.

[0212] First, a layer-by-layer differential scanning method is used to calculate the hidden states of the target large model under the original sample input and adversarial sample input, in order to reveal the response shift of the network to anomalous inputs. The calculation formula is detailed in the above embodiment and will not be repeated here. For example, a difference threshold of 0.35 can be set, and neural network layers exceeding the threshold are included in the target pool that requires high attention.

[0213] After completing the cross-layer differential state scan, this application introduces a "layered vulnerability index" to aggregate and assess the overall risk of each layer. Then, a three-dimensional heat map can be constructed based on the average index values ​​of different samples to graphically display the intensity of abnormal response and form a "vulnerability hotspot" map.

[0214] Furthermore, to further identify the propagation paths of anomalous behavior caused by vulnerable layers, this application can construct a propagation correlation coefficient matrix based on cross-layer state changes. This matrix reveals the degree of anomalous activation coupling between any two hidden state layers. If a certain layer has a high positive correlation with multiple downstream layers, it can be determined that it possesses the characteristics of a "core lesion." The system further constructs a propagation map to track anomalous paths and records the vulnerability contribution on each path.

[0215] Furthermore, to accurately pinpoint the source neural layer of abnormal behavior, the system performs column-by-column distortion analysis on the weight matrix of high-risk layers. This application uses the cosine similarity of each column vector before and after perturbation as an indicator. If the cosine similarity is less than a certain value, the parameters of that neural layer are considered to be severely affected and need to be included in the "root cause candidate set." This application maps these neural network layers back to the original task structure (such as attention heads, FFN nodes) to complete the three-dimensional fusion localization of "function-structure-abnormality." Finally, a "key neural layer / meta-index table" is formed, serving as direct input for subsequent repair strategies.

[0216] ③ Dual-target dynamic repair technology:

[0217] This application introduces a reinforcement learning-based "state awareness – policy selection – layer-by-layer intervention – performance protection" mechanism to achieve an iterative repair process that optimizes both model safety and performance metrics for identified vulnerable layers and neurons. The scheme includes the following key components:

[0218] First, the various risks and performance indicators of the model before repair are uniformly modeled as a state vector s. t The environmental inputs that constitute the reinforcement learning agent are:

[0219] s t = [Vul1, Vul2, ..., Vul L Acc test Latency] T ;

[0220] This state vector can be dynamically updated, reflecting the global safety and performance status of the model before and after the repair.

[0221] Then, this application designs three types of repair operations for the repair process, which are used to intervene in vulnerable areas layer by layer, as detailed in Table 1 above. The repair operations can be used in combination. In each round, the system selects a combination strategy in the form of [Prune,AddReg], ..., [Prune,Reinit], and updates the status after execution.

[0222] Next, to guide the strategy towards optimizing security while maintaining performance, the system is designed with the following dual-objective reward function:

[0223] R t =0.7×(-Δ∑Vul) l )+0.25×ΔAcc+0.05×(-ΔLatency).

[0224] The first term encourages reduced security risks, the second term encourages improved accuracy, and the third term penalizes increased response overhead. This application uses this function feedback as the core signal for reinforcement learning optimization, training the PPO policy network for long-term action policy optimization.

[0225] Finally, this application employs a progressive repair protocol, intervening only in a small number (1-2) of target layers each time while prioritizing the repair of high-risk hidden state layers. After repair, the state is reassessed, and the next round begins. To prevent damage to the model's core parameters, the system activates a "core layer protection mechanism" during the repair process, skipping operations on sensitive layers such as LayerNorm and Embedding. In multiple iterations, the system converges to a set of optimized weights, completing the final enhanced model output. When the output model meets specified requirements (e.g., overall security risk reduction >60%, accuracy fluctuation <±0.5%, response latency increase <8%), the repair is considered successful, and the current model parameters are frozen as the "enhanced model" output. If the requirements are not met, the system reverts to the previous state and enters the "strategy inversion" channel to correct the repair strategy.

[0226] Furthermore, this application can achieve the corresponding implementation effects with the help of Python language, PyTorch deep learning framework, GPU (Graphics Processing Unit), in-vehicle devices and wristband devices.

[0227] In summary, the embodiments of this application mainly include the following three points:

[0228] 1) Automated adversarial sample generation and evaluation technology: Based on differential analysis, the adversarial sample generation mechanism breaks through the traditional manual red team mode and realizes the autonomous construction of semantic-level attacks such as logical contradictions.

[0229] 2) Neural network layer / meta-level security risk localization technology: By quantifying the internal vulnerable modules of the hidden state spatial distance model, a layer sensitivity heat map and cross-layer propagation path tracking mechanism are established.

[0230] 3) Dual-objective balanced repair technology: It integrates a dynamic optimization framework that combines security hardening and performance protection, and uses a targeted parameter adjustment strategy to solve the performance degradation problem caused by traditional global repair.

[0231] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the semantic question answering model training method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0232] This application also provides a model training device based on semantic question answering. Figure 8 This is a schematic diagram of the structure of the semantic question-answering-based model training device provided in this application, as shown below. Figure 8 As shown, the semantic question-answering-based model training device includes:

[0233] The construction module 801 is used to construct adversarial samples corresponding to the original samples; wherein, the question text information of the adversarial samples is generated based on the question text information of the original samples under the semantic perturbation mechanism.

[0234] The acquisition module 802 is used to acquire the response data of each neural network layer in the semantic question answering model to the original sample and the response data to the adversarial sample after inputting the question text information of the original sample and the question text information of the adversarial sample into the semantic question answering model to be trained, respectively.

[0235] The determination module 803 is used to determine at least one vulnerable layer in multiple neural network layers of a semantic question answering model based on the response data of the original samples and the response data of the adversarial samples.

[0236] Repair module 804 is used to repair the target vulnerable layer in at least one vulnerable layer using a pre-set repair strategy, so as to obtain the repaired target question answering model.

[0237] In some embodiments, the response data includes the hidden state vector output by each neural network layer in the semantic question answering model after the question text information of the original sample and the question text information of the adversarial sample are respectively input into the semantic question answering model;

[0238] Module 803 is specifically used for:

[0239] For each neural network layer, the Euclidean distance between the hidden state vector corresponding to the original sample and the hidden state vector corresponding to the adversarial sample is calculated as the response offset difference; where the response offset difference characterizes the impact of the adversarial sample on the output of the neural network layer compared to the original sample.

[0240] From multiple neural network layers of the semantic question answering model, neural network layers with response offset differences greater than the first value are selected and placed into the target pool;

[0241] The neural network layers in the target pool are treated as at least one vulnerable layer.

[0242] In some embodiments, the apparatus further includes a processing module for:

[0243] For each neural network layer, the mean of the response offset difference corresponding to the neural network layer is calculated after inputting the question text information of the original sample and the question text information of multiple adversarial samples into the semantic question answering model, and is used as the layer vulnerability index of the neural network layer.

[0244] From multiple neural network layers of the semantic question answering model, neural network layers with a vulnerability index greater than the second value are selected and placed into the target pool.

[0245] In some embodiments, the processing module is further configured to:

[0246] Based on the response offset differences corresponding to multiple neural network layers, the propagation correlation coefficient matrix corresponding to the semantic question answering model is determined; where each element in the propagation correlation coefficient matrix represents the degree of correlation between response offsets between any two neural network layers.

[0247] From multiple elements of the propagation correlation coefficient matrix, select multiple target neural network layers whose element values ​​are greater than the third value;

[0248] Based on the hierarchical position of multiple target neural network layers in the semantic question answering model, the abnormal path for the semantic question answering model to respond abnormally to adversarial examples is determined.

[0249] In some embodiments, the repair module 804 is specifically used for:

[0250] Construct a target reward function based on at least one of the hierarchical vulnerability index of the target vulnerable layer, the output accuracy rate of the semantic question-answering model, and the response delay rate of the semantic question-answering model;

[0251] Based on the target reward function, use a pre-set repair strategy to repair the target vulnerable layer in at least one vulnerable layer, and obtain a repaired target question-answering model.

[0252] In some embodiments, the repair module 804 is further specifically configured to:

[0253] Select a preset number of vulnerable layers from at least one vulnerable layer in descending order of the response offset difference of at least one vulnerable layer as the target vulnerable layer;

[0254] Based on the target reward function, use a pre-set repair strategy to repair the target vulnerable layer, and obtain a repaired first model;

[0255] Perform a security test and an accuracy test on the first model;

[0256] After determining that the first model passes the security test and passes the accuracy test, use the first model as the repaired target question-answering model.

[0257] In some embodiments, the repair module 804 is further specifically configured to:

[0258] After determining that the first model fails the security test or fails the accuracy test, perform weighted summation on the model parameters of the first model and the model parameters of the semantic question-answering model to obtain the model parameters of the repaired target question-answering model.

[0259] In some embodiments, the repair module 804 is further specifically configured to:

[0260] Determine the sensitive layers in multiple neural network layers of the semantic question-answering model;

[0261] Select a preset number of vulnerable layers from at least one vulnerable layer other than the sensitive layer as the target vulnerable layer.

[0262] In some embodiments, the construction module 801 is specifically configured to:

[0263] Use a pre-set multi-dimensional perturbation strategy to perform semantic perturbation on the question text information of the original sample, and obtain at least one perturbed question text information;

[0264] Based on at least one perturbed question text information, construct at least one candidate adversarial sample;

[0265] Based on the distribution of question text information in the original sample and at least one candidate adversarial sample, the output distribution difference value between the original sample and at least one candidate adversarial sample is determined; wherein, the output distribution difference value characterizes the degree of distribution difference of the output layer of the semantic question answering model after the question text information in the original sample and the candidate adversarial sample are respectively input into the semantic question answering model.

[0266] From at least one candidate adversarial sample, select candidate adversarial samples whose output distribution difference value is greater than the fourth value, and use them as adversarial samples.

[0267] In some embodiments, the construction module 801 is further specifically used for:

[0268] From at least one candidate adversarial sample, select candidate adversarial samples whose output distribution difference value is greater than the fourth value and whose cosine similarity is greater than the fifth value, and use them as adversarial samples.

[0269] Cosine similarity represents the semantic difference between the question text information of the original sample and the question text information of the candidate adversarial sample.

[0270] The semantic question-answering-based model training device provided in this application, employing the semantic question-answering-based model training method described in the above embodiments, can solve the technical problem in related technologies where large language models output low accuracy when facing adversarial examples. Compared with the prior art, the beneficial effects of the semantic question-answering-based model training device provided in this application are the same as those of the semantic question-answering-based model training method provided in the above embodiments, and other technical features in the semantic question-answering-based model training device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0271] This application provides a semantic question-answering-based model training device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the semantic question-answering-based model training method in the above embodiments.

[0272] The following is for reference. Figure 9 , Figure 9This is a schematic diagram of the structure of the semantic question-answering-based model training device provided in this application, illustrating a structure suitable for implementing the semantic question-answering-based model training device of the embodiments of this application. The semantic question-answering-based model training device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 9 The semantic question-answering-based model training device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0273] like Figure 9 As shown, a semantic question-answering-based model training device may include a processing unit 901 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 903 into a random access memory (RAM) 904. The RAM 904 also stores various programs and data required for the operation of the semantic question-answering-based model training device. The processing unit 901, ROM 902, and RAM 904 are interconnected via a bus 905. An input / output (I / O) interface 906 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 906: input devices 907 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 908 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 903 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows the semantic question-answering-based model training device to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows semantic question-answering-based model training devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0274] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 903, or installed from ROM 902. When the computer program is executed by processing device 901, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0275] The semantic question-answering-based model training device provided in this application, employing the semantic question-answering-based model training method described in the above embodiments, can solve the technical problem in related technologies where large language models output low accuracy when facing adversarial examples. Compared with the prior art, the beneficial effects of the semantic question-answering-based model training device provided in this application are the same as those of the semantic question-answering-based model training method provided in the above embodiments, and other technical features in this semantic question-answering-based model training device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0276] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0277] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0278] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the semantic question-answering-based model training method in the above embodiments.

[0279] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0280] The aforementioned computer-readable storage medium may be included in a semantic question-answering-based model training device; or it may exist independently and not be assembled into a semantic question-answering-based model training device.

[0281] The aforementioned computer-readable storage medium carries one or more programs that, when executed by a semantic question-answering-based model training device, cause the semantic question-answering-based model training device to perform the following steps:

[0282] Construct adversarial samples corresponding to the original samples; wherein, the question text information of the adversarial samples is generated based on the question text information of the original samples under a semantic perturbation mechanism;

[0283] After inputting the question text information of the original sample and the question text information of the adversarial sample into the semantic question answering model to be trained, the response data of each neural network layer in the semantic question answering model to the original sample and the response data to the adversarial sample are obtained respectively.

[0284] Based on the response data of the original sample and the response data of the adversarial sample, at least one vulnerable layer in multiple neural network layers of the semantic question answering model is identified.

[0285] Using a pre-set repair strategy, at least one of the target vulnerable layers in the vulnerable layers is repaired to obtain the repaired target question-answering model.

[0286] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0287] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0288] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0289] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., computer programs) for executing the above-described semantic question-answering-based model training method. This addresses the technical problem in related technologies where large language models produce low accuracy in their output responses when facing adversarial examples. Compared to existing technologies, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the semantic question-answering-based model training method provided in the above embodiments, and will not be elaborated upon here.

[0290] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the semantic question-answering-based model training method described above.

[0291] The computer program product provided in this application can solve the technical problem in related technologies where large language models output low accuracy when facing adversarial examples. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the semantic question-answering-based model training method provided in the above embodiments, and will not be repeated here.

[0292] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

[0293] This application discloses A1, a model training method based on semantic question answering, the method comprising:

[0294] Construct adversarial samples corresponding to the original samples; wherein, the question text information of the adversarial samples is generated based on the question text information of the original samples under a semantic perturbation mechanism;

[0295] After inputting the question text information of the original sample and the question text information of the adversarial sample into the semantic question answering model to be trained, the response data of each neural network layer in the semantic question answering model to the original sample and the response data to the adversarial sample are obtained respectively.

[0296] Based on the response data of the original sample and the response data of the adversarial sample, at least one vulnerable layer in multiple neural network layers of the semantic question answering model is identified.

[0297] Using a pre-set repair strategy, at least one of the target vulnerable layers in the vulnerable layers is repaired to obtain the repaired target question-answering model.

[0298] A2. As described in A1, the response data includes the hidden state vector output by each neural network layer in the semantic question answering model after the question text information of the original sample and the question text information of the adversarial sample are respectively input into the semantic question answering model;

[0299] The step of identifying at least one vulnerable layer in multiple neural network layers of the semantic question answering model based on the response data of the original sample and the response data of the adversarial sample includes:

[0300] For each neural network layer, the Euclidean distance between the hidden state vector corresponding to the original sample and the hidden state vector corresponding to the adversarial sample is calculated as the response offset difference; wherein, the response offset difference characterizes the impact of the adversarial sample on the output of the neural network layer compared to the original sample;

[0301] From the multiple neural network layers of the semantic question answering model, the neural network layers whose response offset difference is greater than a first value are selected and placed into the target pool;

[0302] The neural network layers in the target pool are used as at least one of the vulnerable layers.

[0303] A3. As described in A2, after calculating the Euclidean distance between the hidden state vector corresponding to the original sample and the hidden state vector corresponding to the adversarial sample as the response offset difference, the method further includes:

[0304] For each neural network layer, the mean of the response offset difference corresponding to the neural network layer is calculated after the question text information of the original sample and the question text information of multiple adversarial samples are input into the semantic question answering model, and is used as the layer vulnerability index of the neural network layer.

[0305] From the multiple neural network layers of the semantic question answering model, neural network layers with a vulnerability index greater than a second value are selected and placed into the target pool.

[0306] A4. As described in A2 or A3, after calculating the Euclidean distance between the hidden state vector corresponding to the original sample and the hidden state vector corresponding to the adversarial sample as the response offset difference, the method further includes:

[0307] Based on the response offset differences corresponding to multiple neural network layers, the propagation correlation coefficient matrix corresponding to the semantic question answering model is determined; wherein, each element in the propagation correlation coefficient matrix represents the degree of correlation between response offsets between any two neural network layers.

[0308] From the multiple elements of the propagation correlation coefficient matrix, select multiple target neural network layers whose values ​​are greater than the third value;

[0309] Based on the hierarchical position of the multiple target neural network layers in the semantic question answering model, the abnormal path in which the semantic question answering model responds abnormally to the adversarial example is determined.

[0310] A5. The method described in any one of A1 to A3, wherein the step of repairing at least one target vulnerable layer in the vulnerable layer using a pre-set repair strategy to obtain a repaired target question-answering model includes:

[0311] A target reward function is constructed based on at least one of the hierarchical vulnerability index of the target vulnerability layer, the output accuracy of the semantic question answering model, and the response latency of the semantic question answering model;

[0312] Based on the target reward function, a pre-set repair strategy is used to repair at least one of the target vulnerable layers to obtain the repaired target question-answering model.

[0313] A6. The method described in A5, wherein the step of repairing at least one target vulnerable layer in the vulnerable layer based on the target reward function using a pre-set repair strategy to obtain the repaired target question-answering model includes:

[0314] According to the response offset difference of at least one of the vulnerable layers in descending order, a predetermined number of vulnerable layers are selected from the at least one vulnerable layer as the target vulnerable layer;

[0315] Based on the target reward function, the target vulnerable layer is repaired using a pre-set repair strategy to obtain the repaired first model;

[0316] Perform security and accuracy tests on the first model;

[0317] After determining that the first model has passed the security test and the accuracy test, the first model will be used as the repaired target question-answering model.

[0318] A7. The method as described in A6, after performing security and accuracy tests on the first model, the method further includes:

[0319] After determining that the first model fails the security test or the accuracy test, the model parameters of the first model and the model parameters of the semantic question answering model are weighted and summed to obtain the model parameters of the repaired target question answering model.

[0320] A8. The method as described in A6, wherein selecting a predetermined number of vulnerable layers from at least one of the vulnerable layers as the target vulnerable layers includes:

[0321] Identify the sensitive layers in the multiple neural network layers of the semantic question answering model;

[0322] From at least one of the vulnerable layers other than the sensitive layer, a predetermined number of vulnerable layers are selected as the target vulnerable layers.

[0323] A9. The method described in any one of A1 to A3, wherein constructing the adversarial sample corresponding to the original sample includes:

[0324] The original sample's question text information is semantically perturbed using a pre-set multi-dimensional perturbation strategy to obtain at least one perturbed question text information.

[0325] Based on at least one of the perturbated question text information, at least one candidate adversarial sample is constructed;

[0326] Based on the distribution of question text information between the original sample and at least one of the candidate adversarial samples, the output distribution difference value corresponding to the original sample and at least one of the candidate adversarial samples is determined; wherein, the output distribution difference value characterizes the degree of distribution difference of the output layer of the semantic question answering model after the question text information in the original sample and the candidate adversarial samples are respectively input into the semantic question answering model;

[0327] From at least one of the candidate adversarial samples, candidate adversarial samples with an output distribution difference value greater than a fourth value are selected as the adversarial samples.

[0328] A10. The method as described in A9, wherein selecting candidate adversarial samples from at least one candidate adversarial sample whose output distribution difference value is greater than a fourth value, as the adversarial sample, includes:

[0329] From at least one of the candidate adversarial samples, candidate adversarial samples with an output distribution difference value greater than the fourth value and a cosine similarity greater than the fifth value are selected as the adversarial samples;

[0330] The cosine similarity represents the semantic difference between the question text information of the original sample and the question text information of the candidate adversarial sample.

[0331] This application also discloses B1, a model training device based on semantic question answering, the device comprising:

[0332] A construction module is used to construct adversarial samples corresponding to the original samples; wherein, the question text information of the adversarial samples is generated based on the question text information of the original samples under a semantic perturbation mechanism;

[0333] The acquisition module is used to acquire the response data of each neural network layer in the semantic question answering model to the original sample and the response data to the adversarial sample after inputting the question text information of the original sample and the question text information of the adversarial sample into the semantic question answering model to be trained.

[0334] A determination module is used to determine at least one vulnerable layer among multiple neural network layers of the semantic question answering model based on the response data of the original sample and the response data of the adversarial sample.

[0335] The repair module is used to repair at least one of the target vulnerable layers in the vulnerable layers using a pre-set repair strategy, so as to obtain the repaired target question-answering model.

[0336] B2. The apparatus as described in B1, wherein the response data includes the hidden state vector output by each neural network layer in the semantic question answering model after the question text information of the original sample and the question text information of the adversarial sample are respectively input into the semantic question answering model;

[0337] The determining module is specifically used for:

[0338] For each neural network layer, the Euclidean distance between the hidden state vector corresponding to the original sample and the hidden state vector corresponding to the adversarial sample is calculated as the response offset difference; wherein, the response offset difference characterizes the impact of the adversarial sample on the output of the neural network layer compared to the original sample;

[0339] From the multiple neural network layers of the semantic question answering model, the neural network layers whose response offset difference is greater than a first value are selected and placed into the target pool;

[0340] The neural network layers in the target pool are used as at least one of the vulnerable layers.

[0341] B3. The apparatus as described in B2, further comprising: a processing module, configured to:

[0342] For each neural network layer, the mean of the response offset difference corresponding to the neural network layer is calculated after the question text information of the original sample and the question text information of multiple adversarial samples are input into the semantic question answering model, and is used as the layer vulnerability index of the neural network layer.

[0343] From the multiple neural network layers of the semantic question answering model, neural network layers with a vulnerability index greater than a second value are selected and placed into the target pool.

[0344] B4. The apparatus as described in B2 or B3, wherein the processing module is further configured to:

[0345] Based on the response offset differences corresponding to multiple neural network layers, the propagation correlation coefficient matrix corresponding to the semantic question answering model is determined; wherein, each element in the propagation correlation coefficient matrix represents the degree of correlation between response offsets between any two neural network layers.

[0346] From the multiple elements of the propagation correlation coefficient matrix, select multiple target neural network layers whose values ​​are greater than the third value;

[0347] Based on the hierarchical position of the multiple target neural network layers in the semantic question answering model, the abnormal path in which the semantic question answering model responds abnormally to the adversarial example is determined.

[0348] B5. The apparatus as described in any one of B1 to B3, wherein the repair module is specifically used for:

[0349] A target reward function is constructed based on at least one of the hierarchical vulnerability index of the target vulnerability layer, the output accuracy of the semantic question answering model, and the response latency of the semantic question answering model;

[0350] Based on the target reward function, a pre-set repair strategy is used to repair at least one of the target vulnerable layers to obtain the repaired target question-answering model.

[0351] B6. The apparatus as described in B5, wherein the repair module is further specifically used for:

[0352] According to the response offset difference of at least one of the vulnerable layers in descending order, a predetermined number of vulnerable layers are selected from the at least one vulnerable layer as the target vulnerable layer;

[0353] Based on the target reward function, the target vulnerable layer is repaired using a pre-set repair strategy to obtain the repaired first model;

[0354] Perform security and accuracy tests on the first model;

[0355] After determining that the first model has passed the security test and the accuracy test, the first model will be used as the repaired target question-answering model.

[0356] B7. The apparatus as described in B6, wherein the repair module is further specifically used for:

[0357] After determining that the first model fails the security test or the accuracy test, the model parameters of the first model and the model parameters of the semantic question answering model are weighted and summed to obtain the model parameters of the repaired target question answering model.

[0358] B8. The apparatus as described in B6, wherein the repair module is further specifically used for:

[0359] Identify the sensitive layers in the multiple neural network layers of the semantic question answering model;

[0360] From at least one of the vulnerable layers other than the sensitive layer, a predetermined number of vulnerable layers are selected as the target vulnerable layers.

[0361] B9. The apparatus as described in any one of B1 to B3, wherein the construction module is specifically used for:

[0362] The original sample's question text information is semantically perturbed using a pre-set multi-dimensional perturbation strategy to obtain at least one perturbed question text information.

[0363] Based on at least one of the perturbated question text information, at least one candidate adversarial sample is constructed;

[0364] Based on the distribution of question text information between the original sample and at least one of the candidate adversarial samples, the output distribution difference value corresponding to the original sample and at least one of the candidate adversarial samples is determined; wherein, the output distribution difference value characterizes the degree of distribution difference of the output layer of the semantic question answering model after the question text information in the original sample and the candidate adversarial samples are respectively input into the semantic question answering model;

[0365] From at least one of the candidate adversarial samples, candidate adversarial samples with an output distribution difference value greater than a fourth value are selected as the adversarial samples.

[0366] B10. The apparatus as described in B9, wherein the construction module is further specifically used for:

[0367] From at least one of the candidate adversarial samples, candidate adversarial samples with an output distribution difference value greater than the fourth value and a cosine similarity greater than the fifth value are selected as the adversarial samples;

[0368] The cosine similarity represents the semantic difference between the question text information of the original sample and the question text information of the candidate adversarial sample.

[0369] This application also discloses C1, a model training device based on semantic question answering, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the model training method based on semantic question answering as described in any one of A1 to A10.

[0370] This application also discloses D1, a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the semantic question-answering-based model training method as described in any one of A1 to A10.

[0371] This application also discloses E1, a computer program product comprising a computer program that, when executed by a processor, implements the steps of a semantic question-answering-based model training method as described in any one of A1 to A10.

Claims

1. A model training method based on semantic question answering, characterized in that, The method includes: Construct adversarial samples corresponding to the original samples; wherein, the question text information of the adversarial samples is generated based on the question text information of the original samples under a semantic perturbation mechanism; After inputting the question text information of the original sample and the question text information of the adversarial sample into the semantic question answering model to be trained, the response data of each neural network layer in the semantic question answering model to the original sample and the response data to the adversarial sample are obtained respectively. Based on the response data of the original sample and the response data of the adversarial sample, at least one vulnerable layer in multiple neural network layers of the semantic question answering model is identified. Using a pre-set repair strategy, at least one of the target vulnerable layers in the vulnerable layers is repaired to obtain the repaired target question-answering model; The response data includes the hidden state vector output by each neural network layer in the semantic question answering model after the question text information of the original sample and the question text information of the adversarial sample are respectively input into the semantic question answering model; The step of identifying at least one vulnerable layer in multiple neural network layers of the semantic question answering model based on the response data of the original sample and the response data of the adversarial sample includes: For each neural network layer, the Euclidean distance between the hidden state vector corresponding to the original sample and the hidden state vector corresponding to the adversarial sample is calculated as the response offset difference; wherein, the response offset difference characterizes the impact of the adversarial sample on the output of the neural network layer compared to the original sample; From the multiple neural network layers of the semantic question answering model, the neural network layers whose response offset difference is greater than a first value are selected and placed into the target pool; The neural network layers in the target pool are used as at least one of the vulnerable layers.

2. The method as described in claim 1, characterized in that, After calculating the Euclidean distance between the hidden state vector corresponding to the original sample and the hidden state vector corresponding to the adversarial sample as the response offset difference, the method further includes: For each neural network layer, the mean of the response offset difference corresponding to the neural network layer is calculated after the question text information of the original sample and the question text information of multiple adversarial samples are input into the semantic question answering model, and is used as the layer vulnerability index of the neural network layer. From the multiple neural network layers of the semantic question answering model, neural network layers with a vulnerability index greater than a second value are selected and placed into the target pool.

3. The method as described in claim 1 or 2, characterized in that, After calculating the Euclidean distance between the hidden state vector corresponding to the original sample and the hidden state vector corresponding to the adversarial sample as the response offset difference, the method further includes: Based on the response offset differences corresponding to multiple neural network layers, the propagation correlation coefficient matrix corresponding to the semantic question answering model is determined; wherein, each element in the propagation correlation coefficient matrix represents the degree of correlation between response offsets between any two neural network layers. From the multiple elements of the propagation correlation coefficient matrix, select multiple target neural network layers whose values ​​are greater than the third value; Based on the hierarchical position of the multiple target neural network layers in the semantic question answering model, the abnormal path in which the semantic question answering model responds abnormally to the adversarial example is determined.

4. The method as described in claim 1, characterized in that, The step of repairing at least one target vulnerable layer in the vulnerable layer using a pre-set repair strategy to obtain a repaired target question-answering model includes: A target reward function is constructed based on at least one of the hierarchical vulnerability index of the target vulnerability layer, the output accuracy of the semantic question answering model, and the response latency of the semantic question answering model; Based on the target reward function, a pre-set repair strategy is used to repair at least one of the target vulnerable layers to obtain the repaired target question-answering model.

5. The method as described in claim 4, characterized in that, The process of repairing at least one of the vulnerable layers based on the target reward function using a pre-set repair strategy to obtain a repaired target question-answering model includes: According to the response offset difference of at least one of the vulnerable layers in descending order, a predetermined number of vulnerable layers are selected from the at least one vulnerable layer as the target vulnerable layer; Based on the target reward function, the target vulnerable layer is repaired using a pre-set repair strategy to obtain the repaired first model; Perform security and accuracy tests on the first model; After determining that the first model has passed the security test and the accuracy test, the first model will be used as the repaired target question-answering model.

6. A model training device based on semantic question answering, characterized in that, The device includes: A construction module is used to construct adversarial samples corresponding to the original samples; wherein, the question text information of the adversarial samples is generated based on the question text information of the original samples under a semantic perturbation mechanism; The acquisition module is used to acquire the response data of each neural network layer in the semantic question answering model to the original sample and the response data to the adversarial sample after inputting the question text information of the original sample and the question text information of the adversarial sample into the semantic question answering model to be trained. A determination module is used to determine at least one vulnerable layer among multiple neural network layers of the semantic question answering model based on the response data of the original sample and the response data of the adversarial sample. The repair module is used to repair at least one of the target vulnerable layers in the vulnerable layers using a pre-set repair strategy, so as to obtain the repaired target question answering model. The response data includes the hidden state vector output by each neural network layer in the semantic question answering model after the question text information of the original sample and the question text information of the adversarial sample are respectively input into the semantic question answering model; The step of identifying at least one vulnerable layer in multiple neural network layers of the semantic question answering model based on the response data of the original sample and the response data of the adversarial sample includes: For each neural network layer, the Euclidean distance between the hidden state vector corresponding to the original sample and the hidden state vector corresponding to the adversarial sample is calculated as the response offset difference; wherein, the response offset difference characterizes the impact of the adversarial sample on the output of the neural network layer compared to the original sample; From the multiple neural network layers of the semantic question answering model, the neural network layers whose response offset difference is greater than a first value are selected and placed into the target pool; The neural network layers in the target pool are used as at least one of the vulnerable layers.

7. A model training device based on semantic question answering, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the semantic question-answering-based model training method as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the semantic question-answering-based model training method as described in any one of claims 1 to 5.

9. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the semantic question-answering-based model training method as described in any one of claims 1 to 5.

Images