Question and answer processing method based on early stop judgment and multi-head decoding

Abstract

The invention discloses a question and answer processing method based on early stop judgment and multi-head decoding, and relates to the technical field of large model question and answer processing, and the method comprises the steps: obtaining question data input by a user on a human-computer interaction platform, inputting the question data into a preset large model for processing, and obtaining answer data corresponding to the question data; the preset large model comprises a hidden layer, a self-att layer and a FeedFroward layer, and training is carried out by setting an early stop judgment mechanism and a multi-head decoding mechanism. According to the scheme, the calculation amount is reduced and the training efficiency is improved through an early stop judgment mechanism, a multi-head decoding mechanism and an optimized decoding and reasoning stage, so that the training efficiency of the preset large model can be improved, and meanwhile, the training precision of the preset large model is higher due to the reduction of the calculation amount; and obtaining answer data closer to the question data of the user.

Description

Background Art

[0002] Large model training refers to training a complex neural network model through big data. The values ​​of weights and biases in the network are determined through training with a large amount of data so that it can adapt to specific functions. In the training process of large models, the parameters of the model are usually randomly initialized and gradually converge to appropriate values ​​as the training progresses to capture the complex patterns and relationships in the input data. However, the training of large models usually requires a lot of computing resources and time, which limits its widespread promotion in practical applications.

[0003] Existing large model training acceleration methods mainly include distributed training and mixed precision training, but these methods mainly focus on reducing computing and memory resource requirements, and relatively less on optimization in the decoding and inference stages. In addition, the autoregressive decoding mechanism is widely used in large model inference, but its word-by-word output token generation method increases the amount of computation and is time-consuming. As a result, users also get answers at a lower speed when asking questions. Summary of the invention

[0004] The technical problem to be solved by the present invention is to address the deficiencies of the prior art and specifically provide a question-answering processing method based on early stop judgment and multi-head decoding, which is as follows:

[0005] 1) In the first aspect, the present invention provides a question-answering processing method based on early stopping judgment and multi-head decoding, and the specific technical solution is as follows:

[0006] Obtaining question data input by the user on the human-computer interaction platform, inputting the question data into a preset large model for processing, and obtaining answer data corresponding to the question data;

[0007] The preset large model includes a hidden layer, a self-attack layer and a FeedFroward layer, and is trained by setting an early stopping judgment mechanism and a multi-head decoding mechanism.

[0008] The beneficial effects of the question-answering processing method based on early stopping judgment and multi-head decoding provided by the present invention are as follows:

[0009] This solution optimizes the decoding and reasoning stages through the early stopping judgment mechanism and the multi-head decoding mechanism, reduces the amount of calculation and improves the training efficiency. The above method can improve the training efficiency of the preset large model. At the same time, the training accuracy of the preset large model can be higher due to the reduction in the amount of calculation, and the answer data that is closer to the user's question data can be obtained.

[0010] Based on the above solution, the present invention can also be improved as follows.

[0011] Furthermore, the calculation logic of the hidden layer is:

[0012] x=t*W T ;

[0013] Among them, t is the input training data, W is the weight matrix of the hidden layer, T is the transpose, and x is the output result of the hidden layer.

[0014] Furthermore, the calculation logic of the self-att layer is:

[0015]

[0016] Where q = x*W q T , k=x*W k T , v = x * W v T , W q , W k , W v are three different weight matrices, q, k, and v represent the output results of the three linear calculation layers respectively.

[0017] Furthermore, the calculation logic of the FeedForward layer is:

[0018]

[0019] Where, f = x*W f T , W f is the weight matrix of the FeedForward layer, and e is a natural constant.

[0020] 2) In the second aspect, the present invention also provides a question-answering processing system based on early stopping judgment and multi-head decoding, and the specific technical solution is as follows:

[0021] The acquisition module is used to: acquire question data input by the user on the human-computer interaction platform, input the question data into a preset large model for processing, and obtain answer data corresponding to the question data;

[0022] The preset large model includes a hidden layer, a self-attack layer and a FeedFroward layer, and is trained by setting an early stopping judgment mechanism and a multi-head decoding mechanism.

[0023] Based on the above solution, the present invention can also be improved as follows.

[0024] Furthermore, the calculation logic of the hidden layer is:

[0025] x=t*W T ;

[0026] Among them, t is the input training data, W is the weight matrix of the hidden layer, T is the transpose, and x is the output result of the hidden layer.

[0027] Furthermore, the calculation logic of the self-att layer is:

[0028]

[0029] Where q = x*W q T , k=x*W k T , v = x * W v T , W q , W k , W v are three different weight matrices, q, k, and v represent the output results of the three linear calculation layers respectively.

[0030] Furthermore, the calculation logic of the FeedForward layer is:

[0031]

[0032] Where, f = x*W f T , W f is the weight matrix of the FeedForward layer, and e is a natural constant.

[0033] 3) In a third aspect, the present invention further provides an electronic device, comprising a processor, wherein the processor is coupled to a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the electronic device implements any of the above methods.

[0034] 4) In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor so that a computer implements any of the above methods.

[0035] It should be noted that the beneficial effects achieved by the technical solutions of the second to fourth aspects of the present invention and the corresponding possible implementation methods can be found in the above-mentioned technical effects of the first aspect and its corresponding possible implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0037] Figure 1 A flowchart of a question-answering processing method based on early stopping judgment and multi-head decoding according to an embodiment of the present invention;

[0038] Figure 2 The present invention is a structural framework diagram of an electronic device. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0040] like Figure 1 As shown, a question-answering processing method based on early stopping judgment and multi-head decoding in an embodiment of the present invention includes the following steps:

[0041] S1, obtaining question data input by the user on the human-computer interaction platform, inputting the question data into a preset large model for processing, and obtaining answer data corresponding to the question data;

[0042] The preset large model includes a hidden layer, a self-attack layer and a FeedFroward layer, and is trained by setting an early stopping judgment mechanism and a multi-head decoding mechanism.

[0043] The beneficial effects of the question-answering processing method based on early stopping judgment and multi-head decoding provided by the present invention are as follows:

[0044] This solution optimizes the decoding and reasoning stages through the early stopping judgment mechanism and the multi-head decoding mechanism, reduces the amount of calculation and improves the training efficiency. The above method can improve the training efficiency of the preset large model. At the same time, the training accuracy of the preset large model can be higher due to the reduction in the amount of calculation, and the answer data that is closer to the user's question data can be obtained.

[0045] It should be noted that the human-computer interaction platform refers to the IA platform of any website or the intelligent experience AI platform in any experience center. The above-mentioned human-computer interaction platform includes the human-computer interaction interface, that is, the communication media or means between humans and computer systems, and is a platform for two-way information exchange of various symbols and actions between humans and computers.

[0046] Question data refers to the question text entered by the user in the input box on the human-computer interaction interface;

[0047] Answer data refers to the text content related to the question data that is output after the question data is parsed and processed by a preset large model.

[0048] For the input problem data, preprocessing is required to ensure the uniformity and identifiability of the data. The preprocessing process is:

[0049] In order to support the compatibility of the preset large model, there is no limit on the number of characters corresponding to the input question data, and the text content in paragraph units is allowed to be input. Therefore, it is necessary to analyze and optimize the input question data first, that is, determine whether the input question data contains at least one paragraph. If so, segment each paragraph, and divide the segmentation results according to the original order of the input question data, determine the keywords corresponding to each segmentation result, integrate the keywords obtained from each segmentation result, obtain the first keyword set, extract keywords in the entire paragraph as a unit, obtain the second keyword set, determine the first semantic representation corresponding to the first keyword set, determine the second semantic representation corresponding to the second keyword set (through the historical database, search for sentences corresponding to the keywords in the historical database, or search through the target keywords with keyword similarity and correlation not less than 85%), calculate the similarity of the first semantic representation and the second semantic representation, if the similarity exceeds the preset similarity, select the set with fewer characters in the first keyword set and the second keyword set as the input data of the preset large model, and perform subsequent analysis and processing based on the input data. If the similarity is lower than the preset similarity, the first keyword set is used as the input data of the preset large model. The purpose of the above process is to reduce the amount of calculation of the large model. When there is a large difference between the keyword set corresponding to the sentence and the keyword set corresponding to the paragraph, subsequent processing is performed according to the first keyword set with more keywords to maximize the retention of the user's original intention.

[0050] If not, that is, when the input question data is several independent sentences, the correlation and similarity of two consecutive sentences are judged in sequence according to the order of input, and the weighted sum value corresponding to any two sentences is calculated based on the corresponding weights of the correlation and similarity, and the first target sentence whose weighted sum value exceeds the preset weighted sum value is extracted for keywords, and the second target sentence with the lowest correlation with other second target sentences is determined in the second target sentence whose weighted sum value is lower than the preset weighted sum value, and the correlation between the second target sentence and the first target sentence is calculated, and if the calculated result of the correlation with a certain sentence exceeds the preset value, the second target sentence is merged into the first target sentence. The keywords corresponding to all the first target sentences are used as the input of the preset large model.

[0051] Furthermore, the calculation logic of the hidden layer is:

[0052] x=t*W T ;

[0053] Among them, t is the input training data, W is the weight matrix of the hidden layer, T is the transpose, and x is the output result of the hidden layer.

[0054] Furthermore, the calculation logic of the self-att layer is:

[0055]

[0056] Where q = x*W q T , k=x*W k T , v = x * W v T , W q , W k , W v are three different weight matrices, q, k, and v represent the output results of the three linear calculation layers respectively.

[0057] Furthermore, the calculation logic of the FeedForward layer is:

[0058]

[0059] Where, f = x*W f T , W f is the weight matrix of the FeedForward layer, and e is a natural constant.

[0060] In Example 1, the structure of the model is first set as follows:

[0061] First, it passes through the hidden layer. The calculation logic of the hidden layer is:

[0062] x=t*W T

[0063] Where t is the input training data, W is the weight matrix of the hidden layer, and T represents the transposition operation of the matrix. The final output x is the vector representation output obtained through the hidden layer.

[0064] The output hidden layer calculation results are divided into three parts:

[0065] 1) Calculated by the self-att layer;

[0066] 2) Calculated by the FeedFroward layer;

[0067] 3) Add the calculated results of 1) and 2);

[0068] These three parts can be expressed by the following formula:

[0069] y=LayerNorm(x+MLP(x)+Attention(x))

[0070] in:

[0071] 1) Attention in the formula represents the self-att layer,

[0072] The specific calculation is:

[0073] q=x*W q T

[0074] k=x*W k T

[0075] v=x*W v T

[0076] Among them, W q , W k , W v They represent three different weight matrices, T represents transpose, and q, k, and v represent the output results of the three linear calculation layers.

[0077] Next, perform matrix operations on q, k, and v:

[0078]

[0079] Among them, d is a hyperparameter, which is set to 64, and att_output is the output result calculated by the self-att layer.

[0080] 2) The MLP in the formula refers to the FeedForward layer in the figure, where the operation logic is:

[0081] f=x*W f T

[0082] Among them, W f is the weight matrix of the FeedForward layer, and f is the output of the current calculation

[0083]

[0084] Among them, e is the natural constant e, and feed_output is the output result of the FeedForward layer

[0085] 3) The calculation process of LayerNorm in the formula is:

[0086]

[0087] Among them, μ is the mean, σ is the variance, g i is the weight matrix of the LayerNorm layer, a i is the input vector, is the final result of the calculation.

[0088] The model structure is as above, where the number of hidden layer structures of the model is 1, and the number of self-att layers and FeedFroward layers of the model are both N.

[0089] Next, the model training is accelerated through the early stopping judgment mechanism and multi-head decoding mechanism:

[0090] 1 Early stop judgment mechanism module:

[0091] 1.1 Data Input Model

[0092] Assume that the data input to the model is in the format of question-answer pairs, as shown below:

[0093] Question: "What is the capital of China?"

[0094] Answer: "The capital of China is Beijing"

[0095] Input representation: Input the question in the question-answer pair represented by id into the model defined above. The input format is [CLS] question [SEP].

[0096] 1.2 Layer-by-layer calculation

[0097] The problem represented by id is first calculated by the hidden layer, and the calculation result will be used to execute the calculation of the self-att layer and the FeedFroward layer. After each layer of the N layers of self-att layer and FeedFroward layer is calculated,

[0098] Decode the output and convert the output result into a token id (denoted as A). Compare the token id with the token id of the real answer (denoted as B)

[0099] Calculate the edit distance to the true answer.

[0100] The definition and calculation of edit distance are as follows:

[0101] For two strings A and B, assuming their lengths are m and n respectively, the edit distance d(A,B) is calculated using a (m+1)×(n+1) matrix D. The element D[i][j] of the matrix D represents the minimum number of edit operations required to convert the first i characters of string A into the first j characters of string B.

[0102] Specific steps:

[0103] 1) Initialize the matrix

[0104] The matrix D[0][0] is initialized to 0, which means the edit distance between empty strings is 0.

[0105] D[i][0]=i(1≤i≤m): represents converting a string of length i into an empty string, which requires i deletion operations.

[0106] D[0][j]=j(1≤j≤n): represents converting a string of length j into an empty string, which requires j deletion operations.

[0107] 2) Recursive calculation

[0108] For 1≤i≤m and 1≤j≤n, depending on whether A[i] (the i-th character of string A) and B[j] (the j-th character of string B) are equal, perform the following operations respectively:

[0109] If A[i]=B[j], then D[i][j]=D[i-1][j-1]: the current character has been matched and no additional editing operation is required.

[0110] If A[i]≠B[j], then D[i][j]=min{D[i-1][j]+1,D[i][j-1]+1,D[i-1][j-1]+1}:

[0111] D[i-1][j]+1: means deleting A[i] so that the first i-1 characters of A match the first j characters of B.

[0112] D[i][j-1]+1: means inserting B[j] after the i-th character of A, so that the first i characters of A match the first j-1 characters of B and then match B[j].

[0113] D[i-1][j-1]+1: means replacing A[i] with B[j] so that the first i-1 characters of A match the first j-1 characters of B.

[0114] The edit distance between A and B that needs to be calculated is d(A,B), which is the lower right corner element D[m][n] of matrix D.

[0115] 1.3 Early Stopping Decision

[0116] Set a threshold α, where α is a hyperparameter. If the edit distance of the current layer is less than α, stop calculating the subsequent layers of the question-answer pair, and stop training the current input data at the current layer. Do not complete the N-layer model calculations, and continue training the next question-answer pair. In this way, on the one hand, since the lengths of the question-answer pairs in the training data are not all the same, the N-layer network of the model does not have to be completed for shorter question-answer pairs. If the knowledge of this question-answer pair has been fully learned based on the previous judgment, there is no need to complete the entire N-layer model calculation. The learning of the current question-answer pair data can be terminated at the current layer, so as to prevent the remaining network layers from continuing to learn when the learning is sufficient, resulting in overfitting. At the same time, without completing the calculation, the model calculation amount can be reduced, the loss of training resources can be reduced, and the important point is that it can greatly reduce the training time of the model.

[0117] 1.4 Continue training

[0118] If the early stopping condition is not met, the calculation of the next layer continues. When all question-answer pairs are processed, back propagation and parameter update are performed according to the loss function.

[0119] 2. Multi-head decoding module

[0120] In each network layer of the N-layer model, after the model has learned the question data in the question-answer pair, it needs to generate the corresponding answer based on the question in order to determine whether the learning is complete. In this generation process, we use a multi-head decoding mechanism to accelerate the process. The steps are as follows:

[0121] 2.1. Introduce a multi-head attention mechanism to decompose the decoding process into multiple decoding heads that execute in parallel

[0122] The calculation process is:

[0123]

[0124] in, The output result of the feedward layer of each layer of the network, W1, W2, W3 are weight matrices, and b is the bias matrix.

[0125] The next calculation is:

[0126] α_h=softmax(Q_h*K_h^T / sqrt(d_k))

[0127] Where: d_k is a hyperparameter.

[0128] Next, calculate the context vector:

[0129] c_h=α_h*V_h

[0130] Next, the feedforward neural network calculation is performed:

[0131] The calculation formula is:

[0132] FFN=ReLU(W_4*c_h+b_4)*W_5+b_5

[0133] Where: c_h is the input, W_4, W_5 are weight matrices, b_4, b_5 are bias matrices.

[0134] In the multi-head decoding module, we set the number of decoding heads num_heads to 8, which means that there are 8 such decoding networks for simultaneous decoding, instead of relying on all the character information before the current character for decoding as before. Parallel processing can be performed, which greatly speeds up the decoding speed.

[0135] Through the early stopping judgment mechanism and multi-head decoding mechanism, the original training time can be optimized to a great extent and the training speed of the model can be accelerated.

[0136] In the above embodiments, although the steps are numbered S1, S2, etc., these are only specific embodiments given by the present invention. Those skilled in the art may adjust the execution order of S1, S2, etc. according to actual conditions, which is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0137] The present invention also provides a question-answering processing system based on early stopping judgment and multi-head decoding, and the specific technical solution is as follows:

[0138] The acquisition module is used to: acquire question data input by the user on the human-computer interaction platform, input the question data into a preset large model for processing, and obtain answer data corresponding to the question data;

[0139] The preset large model includes a hidden layer, a self-attack layer and a FeedFroward layer, and is trained by setting an early stopping judgment mechanism and a multi-head decoding mechanism.

[0140] Based on the above solution, the present invention can also be improved as follows.

[0141] Furthermore, the calculation logic of the hidden layer is:

[0142] x=t*W T ;

[0143] Among them, t is the input training data, W is the weight matrix of the hidden layer, T is the transpose, and x is the output result of the hidden layer.

[0144] Furthermore, the calculation logic of the self-att layer is:

[0145]

[0146] Where q = x*W q T , k=x*W k T , v = x * W v T , W q , W k , W v are three different weight matrices, q, k, and v represent the output results of the three linear calculation layers respectively.

[0147] Furthermore, the calculation logic of the FeedForward layer is:

[0148]

[0149] Where, f = x*W f T , W f is the weight matrix of the FeedForward layer, and e is a natural constant.

[0150] It should be noted that the beneficial effects of the question-answering processing system based on early stopping judgment and multi-head decoding provided by the above embodiment are the same as the beneficial effects of the question-answering processing method based on early stopping judgment and multi-head decoding, which will not be repeated here. In addition, when the system provided by the above embodiment realizes its functions, it only takes the division of the above-mentioned functional modules as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to actual conditions to complete all or part of the functions described above. In addition, the system and method embodiments provided by the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0151] like Figure 2 As shown, an electronic device 300 according to an embodiment of the present invention includes a processor 320, the processor 320 is coupled to a memory 310, and the memory 310 stores at least one computer program 330, and the at least one computer program 330 is loaded and executed by the processor 320, so that the electronic device 300 implements any of the above methods, specifically:

[0152] The electronic device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors 320 (Central Processing Units, CPU) and one or more memories 310, wherein the one or more memories 310 store at least one computer program 330, and the at least one computer program 330 is loaded and executed by the one or more processors 320, so that the electronic device 300 implements a question-answering processing method based on early stop judgment and multi-head decoding provided in the above embodiment. Of course, the electronic device 300 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The electronic device 300 may also include other components for realizing device functions, which will not be repeated here.

[0153] A computer-readable storage medium according to an embodiment of the present invention stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that a computer implements any one of the above methods.

[0154] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0155] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs any of the above methods.

[0156] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects and represent the definition of a specific order or sequence. The order of use of similar objects can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0157] Those skilled in the art know that the present invention can be implemented as a system, method or computer program product. Therefore, the present disclosure can be specifically implemented in the following forms, namely: it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.

[0158] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0159] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A question-answering processing method based on early stopping judgment and multi-head decoding, characterized in that: include: Obtaining question data input by the user on the human-computer interaction platform, inputting the question data into a preset large model for processing, and obtaining answer data corresponding to the question data; The preset large model includes a hidden layer, a self-attack layer and a FeedFroward layer, and is trained by setting an early stopping judgment mechanism and a multi-head decoding mechanism.

2. According to claim 1, a question-answering processing method based on early stopping judgment and multi-head decoding is characterized in that: The calculation logic of the hidden layer is: x=t*W T ; Among them, t is the input training data, W is the weight matrix of the hidden layer, T is the transpose, and x is the output result of the hidden layer.

3. The question-answering processing method based on early stopping judgment and multi-head decoding according to claim 1, characterized in that: The calculation logic of the self-att layer is: Where q = x*W q T , k=x*W k T , v = x * W v T , W q , W k , W v are three different weight matrices, q, k, and v represent the output results of the three linear calculation layers respectively.

4. The question-answering processing method based on early stopping judgment and multi-head decoding according to claim 1, characterized in that: The calculation logic of the FeedForward layer is: Where, f = x*W f T , W f is the weight matrix of the FeedForward layer, and e is a natural constant.

5. A question-answering processing system based on early stopping judgment and multi-head decoding, characterized in that: include: The acquisition module is used to: acquire question data input by the user on the human-computer interaction platform, input the question data into a preset large model for processing, and obtain answer data corresponding to the question data; The preset large model includes a hidden layer, a self-attack layer and a FeedFroward layer, and is trained by setting an early stopping judgment mechanism and a multi-head decoding mechanism.

6. A question-answering processing system based on early stopping judgment and multi-head decoding according to claim 5, characterized in that: The calculation logic of the hidden layer is: x=t*W T ; Among them, t is the input training data, W is the weight matrix of the hidden layer, T is the transpose, and x is the output result of the hidden layer.

7. A question-answering processing system based on early stopping judgment and multi-head decoding according to claim 5, characterized in that: The calculation logic of the self-att layer is: Where q = x*W q T , k=x*W k T , v = x * W v T , W q , W k , W v are three different weight matrices, q, k, and v represent the output results of the three linear calculation layers respectively.

8. A question-answering processing system based on early stopping judgment and multi-head decoding according to claim 5, characterized in that: The calculation logic of the FeedForward layer is: Where, f = x*W f T , W f is the weight matrix of the FeedForward layer, and e is a natural constant.

9. An electronic device, characterized in that: The electronic device comprises a processor, the processor is coupled to a memory, at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the electronic device implements the method according to any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that a computer implements the method according to any one of claims 1 to 4.

Images