Fine adjustment method based on fluid mechanics natural language question and answer model

Abstract

The invention discloses a fine tuning method based on a fluid mechanics natural language question and answer model. The fine tuning method comprises the steps of collecting a question and answer corpus data set, preprocessing the data set, constructing a dynamic pressure self-adaptive network, training the dynamic pressure self-adaptive network, testing the dynamic pressure self-adaptive network and adjusting and optimizing parameters. According to the dynamic pressure adaptive network provided by the invention, a network structure is constructed based on a fluid mechanics principle and is used for simulating dynamic flowing, distribution and adjustment processes of information in a neural network, adaptive control and compression of feature information flow are realized by simulating a path adjustment rule of fluid in a pressed environment, and the expression efficiency of the model is improved. The constructed dynamic pressure adaptive network shows excellent performance in a natural language processing scene. By performing low-rank modeling and dynamic compression on the high-dimensional feature representation, the model parameter volume is reduced, the training time is shortened, the GPU video memory is reduced, the operation speed is improved, and the method can be used for natural language question answering.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to natural language models. Technical Background

[0002] Language model fine-tuning involves retraining an existing pre-trained model using a small amount of domain- or task-related data to improve its performance on specific tasks. Large language models, with billions of parameters and powerful semantic understanding and generation capabilities, have been widely used in natural language processing for text generation, question-answering systems, sentiment analysis, and translation. However, the high training cost, poor transfer adaptability, and low fine-tuning efficiency associated with these large parameters have become key bottlenecks in current model optimization.

[0003] With the development of artificial intelligence, deep learning technology continues to evolve, model structures become more complex, and the scale of training data continues to grow. This places higher demands on model adjustability and compression performance under limited computing resources. To address the technical challenges faced by pre-trained models during migration, such as large parameter size, high computing resource consumption, and high fine-tuning costs, efficient parameter fine-tuning has emerged in recent years. This approach freezes most of the original model parameters and only performs low-rank adaptation on a portion of the structure, reducing fine-tuning costs while maintaining the original model's performance.

[0004] Currently, there are several efficient parameter fine-tuning methods:

[0005] Incremental methods enhance existing pre-trained models by adding additional parameters or layers, training only the newly added parameters. Selective fine-tuning methods selectively fine-tune certain layers of the model to optimize performance and reduce computational and storage costs. Reparameterization methods utilize low-rank representations to minimize the number of trainable parameters, offering significant advantages in optimizing training efficiency and reducing computational effort.

[0006] In the field of fine-tuning technology for large language models, an urgent technical problem that needs to be solved is to provide a fine-tuning method with strong dynamic adaptability, good semantic stability, and good interpretability. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to overcome the shortcomings of the above-mentioned existing technologies and provide a fine-tuning method based on a fluid mechanics natural language question-answering model with strong dynamic adaptability, good semantic stability and good interpretability.

[0008] The technical solution adopted to solve the above technical problems consists of the following steps:

[0009] (1) Collecting question-answering corpus datasets

[0010] Through manual annotation and corpus screening methods, professional questions and answers with clear semantic structure and domain characteristics are collected as the question and answer corpus dataset, and the dataset is divided into training set, test set and validation set in the ratio of 7:2:1.

[0011] (2) Preprocessing the dataset

[0012] The collected question-answer corpus dataset is structured cleaned and standardized. The standardization process includes text segmentation and noise symbol removal operations. The word2vec method is used to convert the text corpus into input format.

[0013] (3) Constructing a dynamic pressure adaptive network

[0014] The dynamic pressure adaptive network is composed of a feedforward layer, a linear layer, a multi-head attention layer, a fluid attention layer, a connection layer, a normalization layer, and a fully connected layer. The feedforward layer is connected in series with the linear layer. The output of the linear layer is connected to the input of the multi-head attention layer and the fluid attention layer. The output of the multi-head attention layer and the fluid attention layer is connected to the input of the connection layer. The output of the connection layer is connected in series with the normalization layer and the fully connected layer in sequence.

[0015] (4) Training dynamic pressure adaptive network

[0016] 1) Constructing the loss function

[0017] According to formula (1), the matrix of the i-th input language question and answer after passing through the fluid attention layer is constructed

[0018]

[0019] Among them, α1, α2, and α3 represent the weight coefficients output by the gated attention network to fluid attention sublayer 1, fluid attention sublayer 2, and fluid attention sublayer 3, respectively, and ΔW1, ΔW2, and ΔW3 represent the output matrices of fluid attention sublayer 1, fluid attention sublayer 2, and fluid attention sublayer 3, respectively.

[0020] Construct the loss function L according to formula (2):

[0021]

[0022] Where N represents the number of input language questions and answers, and N is a finite positive integer. represents the matrix after the i-th input language question and answer passes through the multi-head attention layer, Represents the matrix after the i-th input language question and answer passes through the fluid attention layer, i is a finite positive integer, ⊙ represents the Hadamard product, β is a hyperparameter, β∈(0,1).

[0023] 2) Training dynamic pressure adaptive network

[0024] The training set is input into the dynamic pressure adaptive network for training. During the training process, the training parameters are: the learning rate r is 0.00001, the optimizer uses the Adam optimizer, and the training is carried out until the loss function L converges.

[0025] (5) Testing the dynamic pressure adaptive network

[0026] The test set is input into the trained dynamic pressure adaptive network for testing, and the natural language question answering model is output.

[0027] (6) Parameter tuning

[0028] The validation set was input into the trained dynamic pressure adaptive network to evaluate various indicators of the dynamic pressure adaptive network. Bayesian optimization was used to tune the learning rate, weight initialization, and hyperparameters. During the tuning process, accuracy and semantic consistency indicators were used as evaluation criteria.

[0029] In step (3) of the present invention, in constructing a dynamic pressure adaptive network, the fluid attention layer is composed of a fluid attention sublayer 1, a fluid attention sublayer 2, and a fluid attention sublayer 3 connected in parallel.

[0030] The fluid attention sublayer 1 of the present invention is composed of a linear layer 2 and a dimensionality reduction layer 1, an activation function layer 1, and a dimensionality increase layer 1 connected in series in sequence; the fluid attention sublayer 2 is composed of a linear layer 3 and a dimensionality reduction layer 2, an activation function layer 2, and a dimensionality increase layer 2 connected in series in sequence; the fluid attention sublayer 4 is composed of a linear layer 3 and a dimensionality reduction layer 3, an activation function layer 3, and a dimensionality increase layer 3 connected in series in sequence.

[0031] The method for constructing the dimensionality reduction layer 1 of the present invention is as follows:

[0032] The following formula is used to construct the dimensionality reduction layer 1 and output the low-dimensional feature matrix W d1 :

[0033] W d1 =θ1⊙(W×A1)

[0034] Where A1 and θ1 both represent learnable low-dimensional weight matrices with d rows and a columns, W represents the input matrix with d rows and d columns, and ⊙ represents the Hadamard product.

[0035] The construction method of the dimensionality reduction layer 2 and the dimensionality reduction layer 3 is the same as that of the dimensionality reduction layer 1.

[0036] In the method for constructing the dimensionality reduction layer 1 of the present invention, A1 and θ1 both represent a learnable low-dimensional weight matrix with d rows and a columns, the value of d rows is 128 to 2048, and the value of a column is 1 to 32; W represents an input matrix with d rows and d columns, the value of d rows is 128 to 2048, and the value of d columns is 128 to 2048.

[0037] The method for constructing the dimensionality-raising layer 1 of the present invention is as follows:

[0038] The following formula is used to construct the dimensionality-raising layer 1, which constitutes the output fluid attention feature dimension matrix ΔW1 of the fluid attention sub-layer 1:

[0039] ΔW1=W′ d1 ×B

[0040] Among them, W′ d1 Represents the output W of dimensionality reduction layer 1 d1 After the activation function layer, the input matrix of the d-row and a-column dimensionality-raising layer 1, B represents the learnable dimensionality-raising weight matrix of a-row and d-column, and the input matrix W′ of the dimensionality-raising layer 2 and the dimensionality-raising layer 3 d2 , W′ d3 The construction method and the input matrix W′ of the dimension-raising layer 1 d1 same.

[0041] The construction method of the dimensionality-increasing layer 2 and the dimensionality-increasing layer 3 is the same as that of the dimensionality-increasing layer 1, and the construction method of the outputs ΔW2 and ΔW3 of the dimensionality-increasing layer 2 and the dimensionality-increasing layer 3 is the same as that of the output ΔW1 of the dimensionality-increasing layer 1.

[0042] In the method for constructing the dimension-raising layer 1 of the present invention, the B represents a learnable dimension-raising weight matrix with rows a and columns d, the values ​​of rows a and columns d are 1 to 32, and the values ​​of columns d are 128 to 2048. d1 Represents an input matrix with d rows and a columns, where the values ​​in rows range from 128 to 2048 and the values ​​in columns range from 1 to 32.

[0043] The dynamic pressure adaptive network proposed in the present invention constructs a network structure based on the principles of fluid mechanics and introduces a fluid attention mechanism to simulate the dynamic flow, distribution and regulation process of information in a neural network. By simulating the law of fluid path adjustment in a pressurized environment, it realizes adaptive control and compression of feature information flow, thereby improving the expression efficiency and structural flexibility of the model. The constructed dynamic pressure adaptive network shows excellent performance in natural language processing scenarios. By performing low-rank modeling and dynamic compression on high-dimensional feature representations, the volume of model parameters is effectively reduced, the training time is greatly shortened, and the GPU video memory and computing burden are significantly reduced. The dynamic pressure adaptive network shows good stability and convergence speed during training, and can achieve rapid fine-tuning and efficient deployment while maintaining or improving the accuracy of the question-answering model. Its structure is highly versatile and can be flexibly embedded in the existing pre-trained language model framework. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of implementation 1 of the present invention.

[0045] Figure 2 It is a structural diagram of the dynamic pressure adaptive network.

[0046] Figure 3 yes Figure 2 Schematic diagram of the structure of the fluid attention layer. DETAILED DESCRIPTION

[0047] The present invention will be further described in detail below with reference to the accompanying drawings and examples, but the present invention is not limited to these examples.

[0048] Example 1

[0049] The fine-tuning method based on the fluid mechanics natural language question answering model of this embodiment consists of the following steps (see Figure 1 ):

[0050] (1) Collecting question-answering corpus datasets

[0051] Through manual annotation and corpus screening methods, professional questions and answers with clear semantic structure and domain characteristics are collected as a question and answer corpus dataset, and the dataset is divided into training set, test set, and validation set in a ratio of 7:2:1.

[0052] (2) Preprocessing the dataset

[0053] The collected question-answer corpus dataset is structured cleaned and standardized. The standardization process includes text segmentation and noise symbol removal operations, and the word2vec method is used to convert the text corpus into input format.

[0054] (3) Constructing a dynamic pressure adaptive network

[0055] Figure 2 The schematic diagram of the structure of the dynamic pressure adaptive network of this embodiment is given. Figure 2 In the embodiment, the dynamic pressure adaptive network is composed of a feedforward layer, a linear layer, a multi-head attention layer, a fluid attention layer, a connection layer, a normalization layer, and a fully connected layer. The feedforward layer is connected in series with the linear layer. The output end of the linear layer is connected to the input end of the multi-head attention layer and the fluid attention layer. The output end of the multi-head attention layer and the fluid attention layer is connected to the input end of the connection layer. The output end of the connection layer is connected in series with the normalization layer and the fully connected layer in sequence.

[0056] Figure 3 Given Figure 2 Schematic diagram of the structure of the fluid attention layer in [1]. Figure 3 In the embodiment, the fluid attention layer is composed of fluid attention sub-layer 1, fluid attention sub-layer 2 and fluid attention sub-layer 3 in parallel.

[0057] In this embodiment, the fluid attention sublayer 1 is composed of a linear layer 2, a dimensionality reduction layer 1, an activation function layer 1, and a dimensionality increase layer 1, all connected in series. The fluid attention sublayer 2 is composed of a linear layer 3, a dimensionality reduction layer 2, an activation function layer 2, and a dimensionality increase layer 2, all connected in series. The fluid attention sublayer 4 is composed of a linear layer 3, a dimensionality reduction layer 3, an activation function layer 3, and a dimensionality increase layer 3, all connected in series.

[0058] The construction method of the dimensionality reduction layer 1 in this embodiment is as follows:

[0059] The following formula is used to construct the dimensionality reduction layer 1 and output the low-dimensional feature matrix W d1 :

[0060] W d1 =θ1⊙(W×A1)

[0061] Where A1 and θ1 both represent a learnable low-dimensional weight matrix with d rows and a columns, where the value of d rows ranges from 128 to 2048 and the value of a column ranges from 1 to 32. In this embodiment, the value of d rows is 728 and the value of a column is 16. W represents an input matrix with d rows and d columns, where the values ​​of d rows and d columns range from 128 to 2048. In this embodiment, the values ​​of d rows and d columns are 728. ⊙ represents the Hadamard product.

[0062] The construction method of dimensionality reduction layer 2 and dimensionality reduction layer 3 is the same as that of dimensionality reduction layer 1.

[0063] The construction method of the dimensionality-raising layer 1 in this embodiment is as follows:

[0064] The following formula is used to construct the dimensionality-raising layer 1, which constitutes the output fluid attention feature dimension matrix ΔW1 of the fluid attention sub-layer 1:

[0065] ΔW1=W′ d1 ×B

[0066] Among them, W′ d1 Represents the output W of dimensionality reduction layer 1 d1 After the activation function layer, for the input matrix of d rows and a columns of the dimensionality-raising layer 1, B represents the learnable dimensionality-raising weight matrix of a rows and d columns, the value of a row is 1 to 32, and the value of d column is 128 to 2048; in this embodiment, the value of a row is 16, and the value of d column is 728.

[0067] Input matrix W′ of dimensionality-raising layer 2 and dimensionality-raising layer 3 d2 , W′ d3 The construction method and the input matrix W′ of the dimension-raising layer 1 d1 same.

[0068] The construction method of the dimensionality-increasing layer 2 and the dimensionality-increasing layer 3 is the same as that of the dimensionality-increasing layer 1, and the construction method of the outputs ΔW2 and ΔW3 of the dimensionality-increasing layer 2 and the dimensionality-increasing layer 3 is the same as the output ΔW1 of the dimensionality-increasing layer 1.

[0069] (4) Training dynamic pressure adaptive network

[0070] According to formula (1), the matrix of the i-th input language question and answer after passing through the fluid attention layer is constructed

[0071]

[0072] Among them, α1, α2, and α3 represent the weight coefficients output by the gated attention network to the fluid attention sub-layer 1, fluid attention sub-layer 2, and fluid attention sub-layer 3, respectively. ΔW1, ΔW2, and ΔW3 represent the output matrices of the fluid attention sub-layer 1, fluid attention sub-layer 2, and fluid attention sub-layer 3, respectively.

[0073] Construct the loss function L according to formula (2):

[0074]

[0075] Where N represents the number of input language questions and answers, and N is a finite positive integer. represents the matrix after the i-th input language question and answer passes through the multi-head attention layer, represents the matrix after the i-th input language question and answer passes through the fluid attention layer, i is a finite positive integer, ⊙ represents the Hadamard product, β is a hyperparameter, β∈(0,1), and the value of β in this embodiment is 0.5.

[0076] 2) Training dynamic pressure adaptive network

[0077] The training set is input into the dynamic pressure adaptive network for training. During the training process, the training parameters are: the learning rate r is 0.00001, the optimizer uses the Adam optimizer, and the training is carried out until the loss function L converges.

[0078] (5) Testing the dynamic pressure adaptive network

[0079] The test set is input into the trained dynamic pressure adaptive network for testing, and the natural language question answering model is output.

[0080] (6) Parameter tuning

[0081] The validation set was input into the trained dynamic pressure adaptive network to evaluate various indicators of the dynamic pressure adaptive network. Bayesian optimization was used to tune the learning rate, weight initialization, and hyperparameters. During the tuning process, accuracy and semantic consistency indicators were used as evaluation criteria.

[0082] Completed the fine-tuning method for the natural language question-answering model based on fluid mechanics.

[0083] Example 2

[0084] The fine-tuning method based on the fluid mechanics natural language question-answering model of this embodiment consists of the following steps:

[0085] (1) Collecting question-answering corpus datasets

[0086] This step is the same as in Example 1.

[0087] (2) Preprocessing the dataset

[0088] This step is the same as in Example 1.

[0089] (3) Constructing a dynamic pressure adaptive network

[0090] The structure of the dynamic pressure adaptive network is the same as that of Example 1.

[0091] The construction method of the dimensionality reduction layer 1 in this embodiment is as follows:

[0092] The following formula is used to construct the dimensionality reduction layer 1 and output the low-dimensional feature matrix W d1 :

[0093] W d1 =θ1⊙(W×A1)

[0094] Here, A1 and θ1 both represent a learnable low-dimensional weight matrix with d rows and a columns, where the value of d rows ranges from 128 to 2048 and the value of a column ranges from 1 to 32. In this embodiment, the value of d rows is 128 and the value of a column is 1. W represents an input matrix with d rows and d columns, where the values ​​of d rows and d columns range from 128 to 2048. In this embodiment, the values ​​of d rows and d columns are 128. ⊙ represents the Hadamard product.

[0095] The construction method of dimensionality reduction layer 2 and dimensionality reduction layer 3 is the same as that of dimensionality reduction layer 1.

[0096] The construction method of the dimensionality-raising layer 1 in this embodiment is as follows:

[0097] The following formula is used to construct the dimensionality-raising layer 1, which constitutes the output fluid attention feature dimension matrix ΔW1 of the fluid attention sub-layer 1:

[0098] ΔW1=W′ d1 ×B

[0099] Among them, W′ d1 Represents the output W of dimensionality reduction layer 1 d1 After the activation function layer passes through the input matrix of the d-row and a-column dimensionality-raising layer 1, B represents the learnable dimensionality-raising weight matrix with row a and column d. The value of row a ranges from 1 to 32, and the value of column d ranges from 128 to 2048. In this embodiment, the value of row a is 1, and the value of column d is 128.

[0100] The construction method of the dimensionality-increasing layer 2 and the dimensionality-increasing layer 3 is the same as that of the dimensionality-increasing layer 1.

[0101] (4) Training dynamic pressure adaptive network

[0102] 1) Constructing the loss function

[0103] Construct the loss function L according to formula (1):

[0104] The expression of the loss function L is the same as that in Example 1.

[0105] In formula (1), β is a hyperparameter, β∈(0,1), and the value of β in this embodiment is 0.1. The meanings and value ranges of other parameters and variables are the same as those in embodiment 1.

[0106] The other steps of this step are the same as those in Example 1.

[0107] The other steps are the same as those in Example 1. The fine-tuning method of the fluid mechanics natural language question-answering model is completed.

[0108] Example 3

[0109] The fine-tuning method based on the fluid mechanics natural language question-answering model of this embodiment consists of the following steps:

[0110] (1) Collecting question-answering corpus datasets

[0111] This step is the same as in Example 1.

[0112] (2) Preprocessing the dataset

[0113] This step is the same as in Example 1.

[0114] (3) Constructing a dynamic pressure adaptive network

[0115] The structure of the dynamic pressure adaptive network is the same as that of Example 1.

[0116] The construction method of the dimensionality reduction layer 1 in this embodiment is as follows:

[0117] The following formula is used to construct the dimensionality reduction layer 1 and output the low-dimensional feature matrix W d1 :

[0118] W d1 =θ1⊙(W×A1)

[0119] Here, A1 and θ1 both represent a learnable low-dimensional weight matrix with d rows and a columns, where the value of d rows ranges from 128 to 2048 and the value of a column ranges from 1 to 32. In this embodiment, the value of d rows is 2048 and the value of a column is 32. W represents an input matrix with d rows and d columns, where the values ​​of d rows and d columns range from 128 to 2048. In this embodiment, the values ​​of d rows and d columns are 2048. ⊙ represents the Hadamard product.

[0120] The construction method of dimensionality reduction layer 2 and dimensionality reduction layer 3 is the same as that of dimensionality reduction layer 1.

[0121] The construction method of the dimensionality-raising layer 1 in this embodiment is as follows:

[0122] The following formula is used to construct the dimensionality-raising layer 1, which constitutes the output fluid attention feature dimension matrix ΔW1 of the fluid attention sub-layer 1:

[0123] ΔW1=W′ d1 ×B

[0124] Among them, B represents a row and d column learnable dimensional weight matrix, W′ d1 Represents an input matrix with d rows and a columns. The value of row a ranges from 1 to 32, and the value of column d ranges from 128 to 2048. In this embodiment, the value of row a is 32, and the value of column d is 2048.

[0125] The construction method of the dimensionality-increasing layer 2 and the dimensionality-increasing layer 3 is the same as that of the dimensionality-increasing layer 1.

[0126] (4) Training dynamic pressure adaptive network

[0127] 1) Constructing the loss function

[0128] Construct the loss function L according to formula (1):

[0129] The expression of the loss function L is the same as that in Example 1.

[0130] In formula (1), β is a hyperparameter, β∈(0,1), and the value of β in this embodiment is 0.9. The meanings and value ranges of other parameters and variables are the same as those in embodiment 1.

[0131] The other steps of this step are the same as those in Example 1.

[0132] The other steps are the same as those in Example 1. The fine-tuning method of the fluid mechanics natural language question-answering model is completed.

[0133] In order to verify the beneficial effects of the present invention, a computer comparative simulation experiment was conducted using the fine-tuning method based on the fluid mechanics natural language question-answering model of Example 1 of the present invention, a low-rank adjustment method (hereinafter referred to as comparative experiment 1), a bias fine-tuning method (hereinafter referred to as comparative experiment 2), and an adapter fine-tuning method (hereinafter referred to as comparative experiment 3). The accuracy P was calculated according to the following formula:

[0134]

[0135] Among them, TP represents the number of samples that are actually positive and predicted to be positive, TN represents the number of samples that are actually negative and predicted to be negative, FP represents the number of samples that are actually negative and predicted to be positive, and FN represents the number of samples that are actually positive and predicted to be negative.

[0136] The experimental and calculation results are shown in Table 1.

[0137] Table 1 Comparative experimental results of the method of Example 1 and the three methods

[0138] Experimental methods Accuracy P (%) Comparative Experiment 1 86.09 Comparative Experiment 2 85.88 Comparative Experiment 3 72.05 Method of the present invention 87.42

[0139] As can be seen from Table 1, the accuracy of the method of the present invention is improved by 1.31% compared with that of comparative experiment 1, by 1.54% compared with that of comparative experiment 2, and by 15.37% compared with that of comparative experiment 3.

Claims

1. A fine-tuning method based on a fluid mechanics natural language question answering model, characterized by It consists of the following steps: (1) Collecting question-answering corpus datasets Through manual annotation and corpus screening methods, professional questions and answers with clear semantic structure and domain characteristics are collected as a question and answer corpus dataset, and the dataset is divided into training set, test set, and validation set in a ratio of 7:2:

1. (2) Preprocessing the dataset The collected question-answer corpus dataset is structured cleaned and standardized. The standardization process includes text segmentation and noise symbol removal operations, and the word2vec method is used to convert the text corpus into input format. (3) Constructing a dynamic pressure adaptive network The dynamic pressure adaptive network consists of a feedforward layer, a linear layer, a multi-head attention layer, a fluid attention layer, a connection layer, a normalization layer, and a fully connected layer. The feedforward layer is connected in series with the linear layer. The output of the linear layer is connected to the input of the multi-head attention layer and the fluid attention layer. The output of the multi-head attention layer and the fluid attention layer is connected to the input of the connection layer. The output of the connection layer is connected in series with the normalization layer and the fully connected layer in sequence. (4) Training dynamic pressure adaptive network 1) Constructing the loss function According to formula (1), the matrix of the i-th input language question and answer after passing through the fluid attention layer is constructed Among them, α1, α2, and α3 represent the weight coefficients output by the gated attention network to the fluid attention sub-layer 1, fluid attention sub-layer 2, and fluid attention sub-layer 3, respectively. ΔW1, ΔW2, and ΔW3 represent the output matrices of the fluid attention sub-layer 1, fluid attention sub-layer 2, and fluid attention sub-layer 3, respectively. Construct the loss function L according to formula (2): Where N represents the number of input language questions and answers, and N is a finite positive integer. represents the matrix after the i-th input language question and answer passes through the multi-head attention layer, represents the matrix after the i-th input language question and answer passes through the fluid attention layer, i is a finite positive integer, ⊙ represents the Hadamard product, β is a hyperparameter, β∈(0,1); 2) Training dynamic pressure adaptive network The training set is input into the dynamic pressure adaptive network for training. During the training process, the training parameters are: the learning rate r is 0.00001, the optimizer uses the Adam optimizer, and the training is carried out until the loss function L converges; (5) Testing the dynamic pressure adaptive network Input the test set into the trained dynamic pressure adaptive network for testing and output a natural language question answering model; (6) Parameter tuning The validation set was input into the trained dynamic pressure adaptive network to evaluate various indicators of the dynamic pressure adaptive network. Bayesian optimization was used to tune the learning rate, weight initialization, and hyperparameters. During the tuning process, accuracy and semantic consistency indicators were used as evaluation criteria.

2. The fine-tuning method based on the fluid dynamics natural language question-answering model according to claim 1, characterized in that: In step (3) of constructing a dynamic pressure adaptive network, the fluid attention layer is composed of fluid attention sublayer 1, fluid attention sublayer 2, and fluid attention sublayer 3 in parallel.

3. The fine-tuning method based on the fluid mechanics natural language question-answering model according to claim 2, characterized in that: The fluid attention sublayer 1 is composed of a linear layer 2 and a dimensionality reduction layer 1, an activation function layer 1, and a dimensionality increase layer 1 connected in series in sequence; the fluid attention sublayer 2 is composed of a linear layer 3 and a dimensionality reduction layer 2, an activation function layer 2, and a dimensionality increase layer 2 connected in series in sequence; the fluid attention sublayer 4 is composed of a linear layer 3 and a dimensionality reduction layer 3, an activation function layer 3, and a dimensionality increase layer 3 connected in series in sequence.

4. The fine-tuning method based on the fluid mechanics natural language question-answering model according to claim 3 is characterized in that The construction method of the dimensionality reduction layer 1 is as follows: The following formula is used to construct the dimensionality reduction layer 1 and output the low-dimensional feature matrix W d1 : IN d1 =θ1⊙(W×A1) Where A1 and θ1 both represent learnable low-dimensional weight matrices with d rows and a columns, W represents the input matrix with d rows and d columns, and ⊙ represents the Hadamard product. The construction method of the dimensionality reduction layer 2 and the dimensionality reduction layer 3 is the same as that of the dimensionality reduction layer 1.

5. The fine-tuning method based on the fluid mechanics natural language question-answering model according to claim 4, characterized in that: The A1 and θ1 both represent a learnable low-dimensional weight matrix with d rows and a columns, where the values ​​of d rows are 128 to 2048 and the values ​​of a columns are 1 to 32; the W represents an input matrix with d rows and d columns, where the values ​​of d rows are 128 to 2048 and the values ​​of d columns are 128 to 2048.

6. The fine-tuning method based on the fluid mechanics natural language question-answering model according to claim 3 is characterized in that The construction method of the dimensionality-raising layer 1 is as follows: The following formula is used to construct the dimensionality-raising layer 1, which constitutes the output fluid attention feature dimension matrix ΔW1 of the fluid attention sub-layer 1: ΔW1=W′ d1 ×B Among them, W′ d1 Represents the output W of dimensionality reduction layer 1 d1 After the activation function layer, the input matrix of the d-row and a-column dimensionality-raising layer 1, B represents the learnable dimensionality-raising weight matrix of a-row and d-column, and the input matrix W′ of the dimensionality-raising layer 2 and the dimensionality-raising layer 3 d2 , W′ d3 The construction method and the input matrix W′ of the dimension-raising layer 1 d1 same; The construction method of the dimensionality-increasing layer 2 and the dimensionality-increasing layer 3 is the same as that of the dimensionality-increasing layer 1, and the construction method of the outputs ΔW2 and ΔW3 of the dimensionality-increasing layer 2 and the dimensionality-increasing layer 3 is the same as that of the output ΔW1 of the dimensionality-increasing layer 1.

7. The fine-tuning method based on the fluid mechanics natural language question-answering model according to claim 6, characterized in that: The B represents a learnable dimensional weight matrix with row a and column d, where row a has a value of 1 to 32 and column d has a value of 128 to 2048; the W′ d1 Represents an input matrix with d rows and a columns, where the values ​​in rows range from 128 to 2048 and the values ​​in columns range from 1 to 32.

Images