A question and answer processing method based on early stopping judgment and multi-head decoding

Abstract

The application discloses a kind of based on early stop judgment and multi-head decoding question and answer processing method, it is related to big model question and answer processing technical field, method includes: obtaining the question data that user inputs on man-machine interactive platform, the question data is input into preset big model and is handled, obtains the answer data corresponding to the question data;The preset big model includes hidden layer, self-att layer and FeedFroward layer, training is carried out by setting early stop judgment mechanism and multi-head decoding mechanism.This scheme is through early stop judgment mechanism and multi-head decoding mechanism, optimization decoding and reasoning stage, reduce the amount of calculation, improve training efficiency, by the above-mentioned mode can improve the training efficiency of preset big model, simultaneously can be caused by reducing the amount of calculation and the training precision of preset big model is higher, obtains the answer data that question data of user is closer.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of large model question and answer processing, and particularly relates to a question and answer processing method based on early stopping judgment and multi-head decoding. BACKGROUND

[0002] Large model training refers to training a complex neural network model through big data, determining the values of weights and biases in the network through a large amount of data training, so that it can adapt to a specific function. In the training process of a large model, the parameters of the model are usually randomly initialized and gradually converge to appropriate values as the training progresses, in order to capture complex patterns and relationships in the input data. However, the training of a large model usually requires a large amount of computing resources and time, which limits its widespread application in practical applications.

[0003] Existing large model training acceleration methods mainly include distributed training and mixed precision training, etc., but these methods mainly focus on reducing the demand for computing and memory resources, while the optimization in the decoding and inference stage is relatively less. In addition, the autoregressive decoding mechanism is widely used in large model inference, but its word-by-word generation of output tokens increases the amount of calculation and consumes time. Based on this, the speed of the answer obtained by the user when asking questions is also relatively low. SUMMARY

[0004] The technical problem to be solved by the present application is to overcome the deficiencies of the prior art, and specifically provides a question and answer processing method based on early stopping judgment and multi-head decoding, as follows:

[0005] 1) In a first aspect, the present application provides a question and answer processing method based on early stopping judgment and multi-head decoding, and the specific technical solutions are as follows:

[0006] Obtain question data input by a user on a human-computer interaction platform, input the question data into a preset large model for processing, and obtain answer data corresponding to the question data;

[0007] The preset large model includes a hidden layer, a self-attention layer and a FeedFroward layer, and is trained through setting an early stopping judgment mechanism and a multi-head decoding mechanism.

[0008] The question and answer processing method based on early stopping judgment and multi-head decoding provided by the present application has the following beneficial effects:

[0009] The early stopping judgment mechanism and the multi-head decoding mechanism optimize the decoding and inference stage, reduce the amount of calculation, improve the training efficiency, and through the above-mentioned way, the training efficiency of the preset large model can be improved, and at the same time, the training accuracy of the preset large model can be higher due to the reduction of the amount of calculation, and the answer data closer to the question data of the user can be obtained.

[0010] 2) In a second aspect, the present application also provides a question and answer processing system based on early stopping judgment and multi-head decoding, and the specific technical solutions are as follows:

[0011] The acquisition module is configured to acquire question data input by a user on the human-computer interaction platform, input the question data into a preset large model for processing, and obtain answer data corresponding to the question data.

[0012] The preset large model comprises a hidden layer, a self-attention layer, and a FeedFroward layer, and is trained by setting an early stopping judgment mechanism and a multi-head decoding mechanism.

[0013] 3) In a third aspect, the present application also provides an electronic device, which comprises a processor and a memory coupled with the processor, and the memory stores at least one computer program, the at least one computer program is loaded and executed by the processor, so that the electronic device implements any one of the above methods.

[0014] 4) In a fourth aspect, the present application also provides a computer readable storage medium, which stores at least one computer program, and the at least one computer program is loaded and executed by a processor, so that the computer implements any one of the above methods.

[0015] It should be noted that the technical solutions of the second aspect to the fourth aspect of the present application and the corresponding possible implementation manners have the beneficial effects as described above for the first aspect and the corresponding possible implementation manners, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0016] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments, made with reference to the accompanying drawings:

[0017] Figure 1 FIG. 1 is a flowchart of a question and answer processing method based on early stopping judgment and multi-head decoding according to an embodiment of the present application;

[0018] Figure 2 FIG. 4 is a structural framework diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0020] As shown in FIG. 1, a question and answer processing method based on early stopping judgment and multi-head decoding according to an embodiment of the present application comprises the following steps: Figure 1

[0021] ​S1, acquire question data input by a user on a human-computer interaction platform, input the question data into a preset large model for processing, and obtain answer data corresponding to the question data;

[0022] The preset large model comprises a hidden layer, a self-att layer and a FeedFroward layer, and is trained through setting an early stopping judgment mechanism and a multi-head decoding mechanism.

[0023] The question and answer processing method based on early stopping judgment and multi-head decoding has the following beneficial effects:

[0024] The early stopping judgment mechanism and the multi-head decoding mechanism optimize the decoding and reasoning stages, reduce the amount of calculation, and improve the training efficiency.

[0025] It should be noted that the human-computer interaction platform refers to an IA platform of any website or an intelligent experience AI platform in any experience hall. The human-computer interaction platform comprises a human-computer interaction interface, that is, a communication medium or means between a human and a computer system, and is a platform for bidirectional information exchange of various symbols and actions between a human and a computer.

[0026] The question data refers to question text input by a user in an input box on the human-computer interaction interface.

[0027] The answer data refers to text content related to the question data output by the preset large model after analyzing and processing the question data.

[0028] The input question data needs to be preprocessed to ensure the uniformity and recognizability of the data.

[0029] To support the compatibility of the pre-defined large model, there is no limit to the number of characters in the input question data; input text content in paragraph form is allowed. Therefore, the input question data needs to be analyzed and optimized first. This involves determining whether the input question data contains at least one paragraph. If so, each paragraph is segmented, and the segmentation results are divided according to the original order of the input question data. Keywords corresponding to each segmentation result are determined, and these keywords are integrated to obtain a first keyword set. Keywords are extracted from the entire paragraph to obtain a second keyword set. A first semantic representation corresponding to the first keyword set is determined, and a second semantic representation corresponding to the second keyword set is determined (this can be done by searching a historical database for sentences corresponding to the keywords, or by searching for target keywords with a similarity and relevance of at least 85%). The similarity between the first and second semantic representations is calculated. If the similarity exceeds a preset similarity, the set with fewer characters in both the first and second keyword sets is selected as the input data for the pre-defined large model, and subsequent analysis and processing are performed based on this input data. If the similarity is lower than the preset similarity, the first keyword set is used as the input data for the pre-defined large model. The purpose of the above process is to reduce the computational load of large models, and when there is a significant difference between the keyword set corresponding to a sentence and the keyword set corresponding to a paragraph, subsequent processing is carried out according to the first keyword set with more keywords, so as to preserve the user's original meaning to the greatest extent.

[0030] If not, i.e., the input problem data consists of several independent sentences, then the relevance and similarity of consecutive sentences are determined sequentially according to the input order. Based on the corresponding weights of relevance and similarity, a weighted sum of the values ​​of any two sentences is calculated. Keywords are extracted from the first target sentence whose weighted sum exceeds a preset weighted sum. Among the second target sentences whose weighted sum is lower than the preset weighted sum, the second target sentence with the lowest relevance to other second target sentences is identified. The relevance between this second target sentence and the first target sentence is calculated. If the relevance between this second target sentence and a certain sentence exceeds a preset value, the second target sentence is merged into the first target sentence. All keywords corresponding to the first target sentences are used as input to the preset large model.

[0031] Example 1: The training of the model is accelerated by using an early stopping judgment mechanism and a multi-head decoding mechanism.

[0032] 1. Early Stop Judgment Mechanism Module:

[0033] 1.1 Data Input Model

[0034] Assuming the input data to the model is in question-answer pair format, an example is as follows:

[0035] Question: "What is the capital of China?"

[0036] Answer: "The capital of China is Beijing."

[0037] Input representation: The question in the question-answer pair after id representation is input into the model defined above, and the input format is [CLS] question [SEP].

[0038] 1.2 Layer-by-layer calculation

[0039] The question after id representation first goes through the calculation of the hidden layer, and the calculation result performs the calculation of the self-att layer and the FeedFroward layer. After the calculation of each layer of the N-layer self-att layer and the FeedFroward layer is completed,

[0040] Decoding the output and converting the output result into token id (denoted as A). Token id and token id of the true answer (denoted as B)

[0041] Calculate the edit distance with the true answer.

[0042] The definition and calculation of the edit distance are as follows:

[0043] For two strings A and B, let their lengths be m and n respectively. The edit distance d(A, B) is calculated by a (m+1) x (n+1) matrix D. The element D[i][j] of the matrix D represents the minimum number of editing operations required to convert the first i characters of the string A to the first j characters of the string B.

[0044] Specific steps:

[0045] 1) Initialize the matrix

[0046] Initialize the matrix D[0][0]= 0, which represents the edit distance from the empty string to the empty string as 0.

[0047] D[i][0]= i (1≤i≤m): represents converting a string of length i to an empty string, which requires i deletion operations.

[0048] D[0][j]= j (1≤j≤n): represents converting a string of length j to an empty string, which requires j deletion operations.

[0049] 2) Recursive calculation

[0050] For 1≤i≤m and 1≤j≤n, according to whether A[i] (the ith character of string A) and B[j] (the jth character of string B) are equal, the following operations are performed respectively:

[0051] If A[i] = B[j], then D[i][j] = D[i-1][j-1]: the current character has been matched, no additional editing operation is needed.

[0052] If A[i]≠B[j], then D[i][j] = min{D[i-1][j]+1, D[i][j-1]+1, D[i-1][j-1]+1}:

[0053] D[i-1][j]+1: indicates deleting A[i], making the first i-1 characters of A match the first j characters of B.

[0054] D[i][j-1]+1: indicates inserting B[j] after the i-th character of A, making the first i characters of A match the first j-1 characters of B and then matching B[j].

[0055] D[i-1][j-1]+1: indicates replacing A[i] with B[j], making the first i-1 characters of A match the first j-1 characters of B.

[0056] The editing distance between A and B needs to be calculated, which is d(A, B), that is, the right lower corner element D[m][n] of the matrix D.

[0057] 1.3 Early stopping judgment

[0058] Set a threshold , as a hyperparameter, if the editing distance of the current layer is less than , stop the subsequent layer calculation of the question and answer pair, and stop the training of the current input data at the current layer, not complete the N-layer model calculation, and continue to train the next question and answer pair. In this way, on the one hand, since the lengths of the training data question and answer pairs are not all consistent, for shorter question and answer pairs, the N-layer network of the model does not have to be completely walked through, and if the knowledge of the question and answer pair has been sufficiently learned according to the previous judgment, the remaining network layers do not need to continue to learn, and the learning of the current question and answer pair data can be terminated at the current layer to prevent overfitting caused by the remaining network layers continuing to learn in the case of having learned sufficiently. At the same time, not having to calculate completely can reduce the model calculation amount and reduce the training resource loss. An important point is that it can greatly reduce the training time of the model.

[0059] 1.4 Continue training

[0060] If the early stopping condition is not met, continue the calculation of the next layer. After all question and answer pairs are processed, perform back propagation and parameter update according to the loss function.

[0061] 2. Multi-head decoding module

[0062] In each layer network of the N-layer model, after learning the question data in the question and answer pair, the model needs to generate the corresponding answer according to the question in order to judge whether the learning is completed. In this generation process, we use the multi-head decoding mechanism to accelerate, and the steps are as follows:

[0063] 2.1. Introduce multi-head attention mechanism to decompose the decoding process into multiple parallel decoding heads

[0064] The calculation process is:

[0065] Q_h = +

[0066] K_h = +

[0067] V_h = +

[0068] wherein, is the output result of the feedward layer of each layer network, , , are weight matrices respectively, and b is a bias matrix.

[0069] Next calculation is:

[0070] α_h = softmax(Q_h * K_h^T / sqrt(d_k))

[0071] wherein: d_k is a hyperparameter.

[0072] Next calculate the context vector:

[0073] c_h =α_h * V_h

[0074] Next, perform feedforward neural network calculation:

[0075] The calculation formula is:

[0076] FFN= ReLU(W_4 *c_h + b_4) * W_5+ b_5

[0077] wherein: c_h is the input, W_4, W_5 are weight matrices, and b_4, b_5 are bias matrices.

[0078] In the multi-head decoding module, we set the decoding head number num_heads to 8, which represents such a calculation that 8 decoding networks are used for simultaneous decoding, instead of relying on all the information of the characters before the current character for decoding as before, which can be processed in parallel, greatly speeding up the decoding speed.

[0079] Through the early stopping judgment mechanism and the multi-head decoding mechanism, the original training time consumption can be greatly optimized, and the training speed of the model is accelerated.

[0080] In the above embodiments, although the steps are numbered S1, S2, etc., it is only a specific embodiment given by the present application, and those skilled in the art can adjust the execution order of S1, S2, etc. according to the actual situation, which is within the protection scope of the present application. It can be understood that in some embodiments, some or all of the above embodiments can be included.

[0081] The present application also provides a question and answer processing system based on early stopping judgment and multi-head decoding, and the specific technical solutions are as follows:

[0082] The acquisition module is used for acquiring problem data input by a user on a man-machine interaction platform, inputting the problem data into a preset large model for processing, and obtaining answer data corresponding to the problem data.

[0083] The preset large model includes a hidden layer, a self-att layer and a FeedFroward layer, and is trained by setting an early stopping judgment mechanism and a multi-head decoding mechanism.

[0084] It should be noted that the beneficial effects of the above-mentioned embodiment of the question and answer processing system based on early stopping judgment and multi-head decoding are the same as those of the above-mentioned question and answer processing method based on early stopping judgment and multi-head decoding, and will not be repeated here. In addition, when the system realizes its function, only the division of the above-mentioned functional modules is taken as an example for illustration, and in actual application, the above-mentioned functions can be completed by different functional modules according to the needs, that is, the system is divided into different functional modules according to the actual situation, to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is described in detail in the method embodiment, which will not be repeated here.

[0085] As shown in Figure 2 The electronic device 300 of the embodiment of the present application, the electronic device 300 includes a processor 320, the processor 320 is coupled with a memory 310, the memory 310 stores at least one computer program 330, at least one computer program 330 is loaded and executed by the processor 320, so that the electronic device 300 realizes any one of the above methods, specifically:

[0086] The electronic device 300 can have a large difference due to different configurations or performances, and can include one or more processors 320 (Central Processing Units, CPUs) and one or more memories 310, wherein the one or more memories 310 store at least one computer program 330, the at least one computer program 330 is loaded and executed by the one or more processors 320, so that the electronic device 300 implements the early stopping judgment and multi-head decoding based question and answer processing method provided by the above-mentioned embodiments. Of course, the electronic device 300 can also have a wired or wireless network interface, a keyboard, an input and output interface and the like, so as to perform input and output, and the electronic device 300 can also include other components for realizing the functions of the device, which will not be described here.

[0087] The computer readable storage medium of the embodiment of the application stores at least one computer program, and the at least one computer program is loaded and executed by the processor, so that the computer implements any one of the above methods.

[0088] Optionally, the computer readable storage medium can be a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), a magnetic tape, a floppy disk and an optical data storage device, etc.

[0089] In the exemplary embodiments, a computer program product or computer program is also provided, which includes computer instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the electronic device executes any one of the above methods.

[0090] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application are used to distinguish similar objects, and represent a specific order or sequence. The order of use of similar objects can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described.

[0091] Those skilled in the art know that the present application can be implemented as a system, a method, or a computer program product, therefore, the present disclosure can be embodied in the form of a complete hardware, a complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, which is generally referred to as "circuitry", "module" or "system". In addition, in some embodiments, the present application can also be embodied in the form of a computer program product in one or more computer readable media, which contains computer readable program codes.

[0092] Any combination of one or more computer readable medium can be employed. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0093] Although the embodiments of the present application have been shown and described above, it should be understood that the above-described embodiments are exemplary, and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-described embodiments within the scope of the present application.

Claims

1. A question and answer processing method based on early stopping judgment and multi-head decoding, characterized in that, The method comprises: obtaining question data input by a user on a human-computer interaction platform, inputting the question data into a preset large model for processing to obtain answer data corresponding to the question data; the preset large model comprises a hidden layer, a self-att layer and a FeedFroward layer, and is trained by setting an early stopping judgment mechanism and a multi-head decoding mechanism; further comprising: preprocessing the input question data; the preprocessing process is: determining whether the input question data contains at least one paragraph, if yes, performing paragraph segmentation on each paragraph, dividing the segmentation results according to the original order of the input question data, determining the keywords corresponding to each division result, integrating the keywords obtained from each division result to obtain a first keyword set, extracting keywords in units of entire paragraphs to obtain a second keyword set, determining a first semantic representation corresponding to the first keyword set, and determining a second semantic representation corresponding to the second keyword set, i.e. searching for sentences corresponding to the keywords in the historical database, or searching for target keywords with a keyword similarity and a correlation degree of no less than 85%, calculating the similarity of the first semantic representation and the second semantic representation, if the similarity exceeds a preset similarity, selecting the set with fewer characters in the first keyword set and the second keyword set as the input data of the preset large model, and performing subsequent analysis and processing based on the input data; if the similarity is lower than the preset similarity, the first keyword set is used as the input data of the preset large model; if not, i.e. when the input question data is independent of several sentences, the relevance and similarity of the two consecutive sentences are sequentially judged according to the input order, and the weighted sum value corresponding to any two sentences is calculated based on the corresponding weights of the relevance and similarity, the first target sentence with a weighted sum value exceeding a preset weighted sum value is extracted, the second target sentence with the lowest correlation degree with other second target sentences is determined in the second target sentence with a weighted sum value lower than the preset weighted sum value, the correlation degree calculation between the second target sentence and the first target sentence is performed, if the correlation degree calculation result between the second target sentence and a certain sentence exceeds a preset value, the second target sentence is merged into the first target sentence; the keywords corresponding to all first target sentences are used as the input of the preset large model.

2. A question and answer processing system based on early stopping judgment and multi-head decoding, adopting a question and answer processing method based on early stopping judgment and multi-head decoding according to claim 1, characterized in that, The method comprises: The acquisition module is configured to: obtain question data input by a user on a human-computer interaction platform, input the question data into a preset large model for processing to obtain answer data corresponding to the question data; the preset large model comprises a hidden layer, a self-att layer and a FeedFroward layer, and is trained by setting an early stopping judgment mechanism and a multi-head decoding mechanism.

3. An electronic device, comprising: The electronic device comprises a processor and a memory, the memory stores at least one computer program, the at least one computer program is loaded and executed by the processor, so that the electronic device realizes the method of claim 1.

4. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by the processor, so that the computer implements the method in claim 1.

Images