Natural language processing task processing method and device, equipment and storage medium

Abstract

The invention provides a natural language processing task processing method and device, a server and a medium. The method comprises the steps of obtaining to-be-processed data of a natural language processing task; inputting the to-be-processed data into a recurrent neural network model so as to determine a historical weight or a current weight of historical input data according to a pre-trained deep tree intersection model deployed in the recurrent neural network model, the parameter information output by the deep tree intersection model is adopted to replace the original historical weight or the current weight, so that the demand of resources participating in calculation in the processing process of the to-be-processed data is reduced, the accuracy of data processing of the recurrent neural network model is improved, and the processing efficiency of a natural language processing task is also improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device and storage medium for processing natural language processing tasks. Background Art

[0002] Natural language processing is one of the most important fields in artificial intelligence. Natural language processing is an important technology that reflects language intelligence. It can analyze, understand or generate natural language and realize natural communication between people and machines. Natural language processing tasks include text classification, sentiment analysis, sequence labeling, etc.

[0003] At present, deep learning models have been widely used in natural language processing. The basis of deep learning is the recurrent neural network model (Multi-Layer Perception, MLP). Each layer of the existing MLP is fully connected, and all training parameters of each layer will be applied to all inputs of this layer. The parameters are huge, the computing pressure of the equipment is also high, and the processing efficiency of natural language processing tasks is low. Summary of the invention

[0004] The present application provides a natural language processing task processing method, apparatus, device and storage medium to solve the problem of low processing efficiency in the prior art.

[0005] On the one hand, the present application provides a method for processing a natural language processing task, the method comprising:

[0006] Obtain data to be processed for natural language processing tasks;

[0007] Inputting the data to be processed into a recurrent neural network model to obtain a processing result output by the recurrent neural network model;

[0008] Among them, a pre-trained deep tree intersection model is deployed in the recurrent neural network model, and the deep tree intersection model is used to determine the historical weights of the historical input data of the recurrent neural network model or determine the current weights of the current input data of the recurrent neural network model.

[0009] Optionally, if the deep tree intersection model is used to determine the historical weights of the historical input data of the recurrent neural network model, then the data to be processed is input into the recurrent neural network model to obtain the processing result output by the recurrent neural network model, including:

[0010] Obtaining the current weight of the recurrent neural network model, the current weight corresponding to the recurrent neural network model at any time being the same;

[0011] Determine the historical weight corresponding to the current moment of the recurrent neural network model according to the deep tree intersection model;

[0012] The hidden layer corresponding to the current moment of the recurrent neural network model is used to process the data to be processed, and the data to be processed is corrected according to the historical weight and the current weight to obtain the processing result of the natural language processing task.

[0013] Optionally, if the deep tree intersection model is used to determine the current weight of the current input data of the recurrent neural network model, then the data to be processed is input into the recurrent neural network model to obtain the processing result output by the recurrent neural network model, including:

[0014] Obtaining the historical weights of the recurrent neural network model, the historical weights corresponding to the recurrent neural network model at any time being the same;

[0015] Determine the current weight of the recurrent neural network model at the current moment according to the deep tree intersection model;

[0016] The hidden layer corresponding to the current moment of the recurrent neural network model is used to process the data to be processed, and the data to be processed is corrected according to the historical weight and the current weight to obtain the processing result of the natural language processing task.

[0017] Optionally, determining the current weight corresponding to the recurrent neural network model at the current moment according to the deep tree intersection model includes:

[0018] Inputting the data to be processed into the input node of each tree structure in the deep tree intersection model to obtain multiple leaf node scores of each tree structure;

[0019] Based on the scores of multiple leaf nodes of each tree structure, select the leaf node with the highest score of each leaf node of each tree structure as the target leaf node;

[0020] The parameters corresponding to the target leaf node are used as the current weights corresponding to the recurrent neural network model at the current moment.

[0021] Optionally, inputting the data to be processed into an input node of each tree structure in the deep tree intersection model to obtain multiple leaf node scores of each tree structure includes:

[0022] The data to be processed is input into the input node of each tree structure, and based on linear transformation and Gumbel distribution function, multiple leaf node scores of each tree structure are obtained.

[0023] Optionally, the parameters corresponding to the target leaf nodes of each tree structure include weight parameters and bias parameters; and using the parameters corresponding to the target leaf nodes as the current weights corresponding to the recurrent neural network model at the current moment includes:

[0024] Performing a sum or product operation on the weight parameters corresponding to the target leaf nodes of each tree structure to obtain a first logical parameter;

[0025] Performing a summation or product operation on the deviation parameters corresponding to the target leaf nodes of each tree structure to obtain a second logic parameter;

[0026] According to the first logical parameter and the second logical parameter, a target parameter corresponding to the target leaf node is determined, and the target parameter is used as the current weight corresponding to the recurrent neural network model at the current moment.

[0027] In a second aspect, the present application provides a processing device for a natural language processing task, the device comprising:

[0028] An acquisition module is used to obtain the data to be processed for the natural language processing task;

[0029] A processing module, used for inputting the data to be processed into a recurrent neural network model to obtain a processing result output by the recurrent neural network model;

[0030] Among them, a pre-trained deep tree intersection model is deployed in the recurrent neural network model, and the deep tree intersection model is used to determine the historical weights of the historical input data of the recurrent neural network model or determine the current weights of the current input data of the recurrent neural network model.

[0031] Optionally, if the deep tree intersection model is used to determine the historical weights of the historical input data of the recurrent neural network model, the apparatus further comprises: a determination module;

[0032] The acquisition module is further used to acquire the current weight of the recurrent neural network model, and the current weight corresponding to the recurrent neural network model at any time is the same;

[0033] The determination module is used to determine the historical weight corresponding to the current moment of the recurrent neural network model according to the deep tree intersection model;

[0034] The processing module is used to use the hidden layer corresponding to the current moment of the recurrent neural network model to process the data to be processed, and to correct the data to be processed according to the historical weight and the current weight to obtain the processing result of the natural language processing task.

[0035] Optionally, if the deep tree intersection model is used to determine the current weight of the current input data of the recurrent neural network model, the device further comprises: a determination module;

[0036] The acquisition module is further used to acquire the historical weights of the recurrent neural network model, and the historical weights corresponding to the recurrent neural network model at any time are the same;

[0037] The determination module is used to determine the current weight corresponding to the recurrent neural network model at the current moment according to the deep tree intersection model;

[0038] The processing module is used to use the hidden layer corresponding to the current moment of the recurrent neural network model to process the data to be processed, and to correct the data to be processed according to the historical weight and the current weight to obtain the processing result of the natural language processing task.

[0039] Optionally, the processing module is used to input the data to be processed into the input node of each tree structure in the deep tree intersection model to obtain multiple leaf node scores of each tree structure; based on the multiple leaf node scores of each tree structure, select the leaf node with the highest leaf node score of each tree structure as the target leaf node; and use the parameters corresponding to the target leaf node as the current weight corresponding to the recurrent neural network model at the current moment.

[0040] Optionally, the processing module is used to input the data to be processed into the input node of each tree structure, and obtain multiple leaf node scores of each tree structure based on linear transformation and Gumbel distribution function.

[0041] Optionally, the processing module is used to perform a sum or product operation on the weight parameters corresponding to the target leaf nodes of each tree structure to obtain a first logical parameter;

[0042] Performing a summation or product operation on the deviation parameters corresponding to the target leaf nodes of each tree structure to obtain a second logic parameter;

[0043] The determination module is used to determine the target parameter corresponding to the target leaf node according to the first logical parameter and the second logical parameter, and use the target parameter as the current weight corresponding to the recurrent neural network model at the current moment.

[0044] In a third aspect, the present application provides a processing device for a natural language processing task, the device comprising:

[0045] Memory;

[0046] processor;

[0047] Wherein, the memory stores computer-executable instructions;

[0048] The processor executes the computer-executable instructions stored in the memory to implement the processing method of the natural language processing task as described in the first aspect and various possible implementation methods of the first aspect.

[0049] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement a method for processing a natural language processing task as described in the first aspect and various possible implementations of the first aspect.

[0050] The processing method of the natural language processing task provided in the present application obtains the data to be processed of the natural language processing task; inputs the data to be processed into a recurrent neural network model to determine the historical weights or current weights of the historical input data according to a pre-trained deep tree intersection model deployed in the recurrent neural network model, and then uses the parameter information output by the deep tree intersection model to replace the original historical weights or current weights, thereby reducing the resource requirements involved in the calculation during the processing of the data to be processed, improving the accuracy of the data processing of the recurrent neural network model, and also improving the processing efficiency of the natural language processing task. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0052] Figure 1 An example of the structure of the recurrent neural network model in the prior art provided in this application Figure 1 ;

[0053] Figure 2 Schematic diagram of the process of processing the natural language processing task provided in this application Figure 1 ;

[0054] Figure 3 The structure of the recurrent neural network model in the prior art provided by this application is shown in FIG. Figure 2 ;

[0055] Figure 4 Schematic diagram of the process of processing the natural language processing task provided in this application Figure 2 ;

[0056] Figure 5 Schematic diagram of the structure of the recurrent neural network model provided in this application Figure 1 ;

[0057] Figure 6 The tree structure provided for this application is a three-level single tree full binary structure;

[0058] Figure 7 A two-dimensional deep tree intersection model provided for this application;

[0059] Figure 8 Schematic diagram of the process of processing the natural language processing task provided in this application Figure 3 ;

[0060] Fig. 9 Schematic diagram of the structure of the recurrent neural network model provided in this application Figure 2 ;

[0061] Fig.10 A schematic diagram of the structure of a processing device for a natural language processing task provided in an embodiment of the present application;

[0062] Fig.11 A schematic diagram of the structure of a processing device for a natural language processing task provided in an embodiment of the present application.

[0063] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments made by ordinary technicians in this field under the enlightenment of the embodiments belong to the scope of protection of the present application.

[0065] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0066] Next, let’s first introduce the recurrent neural network model that is widely used in natural language processing tasks in the prior art.

[0067] Figure 1 Structural example of a recurrent neural network model in the prior art provided in the embodiments of the present application Figure 1 .like Figure 1 As shown, the recurrent neural network model structure has an input layer x and a hidden layer y (1) ,y (2) , output layer y (3) , where the number of hidden layers depends on the needs. Figure 1 The recurrent neural network model shown in contains two hidden layers y (1) and (2) .

[0068] like Figure 1 As shown, in the input layer x, x∈{x1,x2,…,x D} is the input value of D feature dimensions, and the data in the input layer passes through two hidden layers y (1) and (2) , in the output layer y (3) Returns K values, namely

[0069] The input values ​​of the D features input in the input layer x are propagated from the input layer to the first hidden layer, and the propagation satisfies the following formula (1):

[0070] y (1) =σ(W (1) ·x+b (1) ) (1)

[0071] Among them, W (1) is the weight parameter of the first hidden layer, b (1) is the bias parameter of the first hidden layer. If the first hidden layer consists of M neurons, then W (1) ∈R D×M , b (1) ∈R 1×M ; σ is the activation function, which is used for nonlinear function transformation.

[0072] In the recurrent neural network model, the first hidden layer y (1) The output value of will be used as the next hidden layer y (2) The input of the second hidden layer y (2) The output satisfies the following formula (2):

[0073] y (2) =σ(W (2) ·y (1) +b (2) ) (2)

[0074] Among them, W (2) is the weight parameter of the second hidden layer, b (2) is the bias parameter of the second hidden layer. If the second hidden layer consists of N neurons, then W (2) ∈R M×N , b (2) ∈R 1×N .

[0075] Then, the second hidden layer y (2) The output value is used as the output layer y (3) The input and output layer values ​​satisfy the following formula (3):

[0076] y (3) =softmax(W (3) ·y (2) + b (3) ) (3)

[0077] Among them, W (3) is the weight parameter of the output layer, b (3) is the bias parameter of the output layer. If K categories are output, then W (3) ∈R N×K , b (2) ∈R 1×K The normalized exponential function softmax is used for multi-classification tasks and can convert the K values ​​of the output layer into a probability distribution ranging from [0,1] and summing to 1.

[0078] Right now Among them, z i The output layer y (3) The i-th value returned, i = 1, 2, 3, ..., K, and

[0079] It should be understood that the weight parameter W of the first hidden layer (1) and the deviation parameter b (1) , the weight parameter W of the second hidden layer (2) and the deviation parameter b (2) , and the weight bias W of the output layer (3) and the deviation parameter b (3) All of them are obtained after the training of the recurrent neural network model is completed.

[0080] In the recurrent neural network model, each neuron in each layer is connected to each neuron in the previous layer. Therefore, the hidden layer and output layer in the recurrent neural network are fully connected layers (FC). All parameters of each layer are applied to the input of this layer to obtain the output value of this layer. Therefore, in the prior art, in the natural language processing process, the number of model parameters is huge, and the number of parameters involved in the calculation is large, the calculation efficiency is low, and a more powerful central processing unit (CPU) is required.

[0081] In view of this, the present application proposes a method for processing natural language processing tasks. The method deploys a pre-trained deep tree intersection model in a recurrent neural network model so that when the recurrent neural network processes data, it uses the parameter information output by the deep tree intersection model to replace the original historical weights or current weights, thereby reducing the resource requirements involved in the calculation during the processing of the data to be processed, improving the accuracy of data processing of the recurrent neural network model, and also improving the processing efficiency of natural language processing tasks.

[0082] The technical solution of the present application is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0083] Figure 2 Schematic diagram of the process of processing a natural language processing task provided in the embodiment of the present application Figure 1 .like Figure 2 As shown, the processing method of the natural language processing task provided by this application includes:

[0084] S101: Obtaining data to be processed for a natural language processing task;

[0085] The data to be processed may be, for example, speech data for model training. The data to be processed may be obtained from a database or through a network. This application does not limit this.

[0086] S102: Input the data to be processed into a recurrent neural network model to obtain a processing result output by the recurrent neural network model.

[0087] Among them, a pre-trained deep tree intersection model is deployed in the recurrent neural network model, and the deep tree intersection model is used to determine the historical weights of the historical input data of the recurrent neural network model or determine the current weights of the current input data of the recurrent neural network model.

[0088] The deep tree intersection model includes multiple tree structures, each of the multiple tree structures includes an input node, a transformation node and a leaf node. The input node of each tree structure is used to input the data to be processed, the transformation node of each tree structure is used to perform linear transformation on the data to be processed, and the leaf node of each tree structure is used to output the leaf node score based on the data after the linear change. The parameter corresponding to the leaf node with the highest score in the leaf nodes of each tree structure can be used as the historical weight of the historical input data of the recurrent neural network model or the current weight of the current input data of the recurrent neural network model.

[0089] It should be understood that the tree structure in the deep tree intersection model has at least three levels. The top is the first level, which serves as the input of the tree structure and is composed of an input node; the middle level can be one level or multiple levels, and each middle level is composed of multiple transformation nodes; the bottom is the last level, which is composed of multiple leaf nodes.

[0090] Figure 3 The structure of the recurrent neural network model in the prior art provided in this embodiment is shown as follows: Figure 2 .like Figure 3 As shown, the recurrent neural network model includes an input layer, an output layer, and multiple hidden layers, and different input moments correspond to different hidden layers.

[0091] The process of outputting data in the recurrent neural network model is as follows: the input layer transmits the input data to the hidden layer corresponding to the input time, the hidden layer processes the input data, and uses the historical weights and current weights to correct the data to obtain the output result. Figure 3 As shown, input data X t After the hidden layer data processing, we get h t , and then use the historical weight W and current weight V to measure h t Correction processing is performed to obtain the corresponding O t .

[0092] It can be understood that the existing historical weight W or current weight V are fixed values, and their determination can be preset or default, for example. In the recurrent neural network model in this embodiment, a deep tree intersection model is deployed, and the deep tree intersection model is used to determine the historical weight W or the current weight V. That is: the deep tree intersection model can determine different historical weights W or current weights V based on the input data, thereby improving the accuracy of the data processing of the recurrent neural network model. Since the data processing is more accurate, the purpose of reducing the resource requirements involved in the calculation can be further achieved.

[0093] The processing method of the natural language processing task provided in this embodiment obtains the data to be processed of the natural language processing task; inputs the data to be processed into a recurrent neural network model to determine the historical weights or current weights of the historical input data according to a pre-trained deep tree intersection model deployed in the recurrent neural network model, and then uses the parameter information output by the deep tree intersection model to replace the original historical weights or current weights, thereby reducing the resource requirements involved in the calculation during the processing of the data to be processed, improving the accuracy of the data processing of the recurrent neural network model, and also improving the processing efficiency of the natural language processing task.

[0094] Figure 4 Schematic diagram of the process of processing a natural language processing task provided in the embodiment of the present application Figure 2 The deep tree intersection model in this embodiment is used to determine the current weight of the current input data of the recurrent neural network model. Figure 4 As shown, the processing method of the natural language processing task provided by this application includes:

[0095] S201: Obtaining data to be processed for a natural language processing task.

[0096] Step S201 is similar to the above step S101 and will not be described again.

[0097] S202: Obtain the historical weights of the recurrent neural network model, where the historical weights corresponding to the recurrent neural network model at any time are the same.

[0098] Figure 5 The structure of the recurrent neural network model provided in this embodiment is shown in FIG. Figure 1 .like Figure 5 As shown, the current weight V in the recurrent neural network model will be determined by the output result of the deep tree intersection model.

[0099] parameter Figure 4 and Figure 5 It can be seen that the historical weight W corresponding to any time is the same, and the value of the historical weight W is fixed. The historical weight W can be obtained according to network training, for example.

[0100] Since the deep tree intersection model in this embodiment is used to determine the current weight; therefore, this step only needs to obtain the historical weight W pre-set in the recurrent neural network model.

[0101] S203: Input the data to be processed into the input node of each tree structure in the deep tree intersection model to obtain multiple leaf node scores of each tree structure.

[0102] S204: Based on the scores of the multiple leaf nodes of each tree structure, select a leaf node with the highest score in each tree structure as a target leaf node.

[0103] S205: Using the parameters corresponding to the target leaf node as the current weight corresponding to the recurrent neural network model at the current moment.

[0104] The parameters corresponding to the target leaf nodes of each tree structure include weight parameters and bias parameters. A possible implementation of the above S203 includes: inputting the data to be processed into the input node of each tree structure, and obtaining multiple leaf node scores of each tree structure based on linear transformation and Gumbel distribution function.

[0105] Figure 6 The tree structure is a three-level single tree full binary structure, such as Figure 6 As shown in , the data of the input node is transmitted to the transformation node through linear transformation and activation function, and the data of the transformation node is transmitted to the leaf node through linear transformation and activation function. Figure 4 In the tree structure shown, the input node is the 0th layer, the transformation node is the first layer, and the leaf node is the second layer. Figure 4 , taking the data to be processed in the natural language processing task as vector data as an example, the method provided in the embodiment of the present application is described in detail.

[0106] In one possible implementation, the transformation node can be represented as The output of the transformation node satisfies the following formula (4):

[0107]

[0108] Where X is the data input to the input node, i represents the transformation layer number of the current node, j represents the position of the current node in the current layer, and t represents the number of the current tree structure. is the weight parameter when the tree structure numbered t is linearly transformed to the jth position of the i-th layer, is the deviation parameter when the tree structure numbered t is linearly transformed to the jth position of the i-th layer.

[0109] In one possible implementation, a leaf node can be represented as The output of the leaf node satisfies the following formula (5):

[0110]

[0111] in, is the value of the transformation node in the previous layer of the leaf node, i represents the layer number of the current node, j represents the position of the current node in the current layer, and t represents the number of the current tree structure. is the weight parameter when the tree structure numbered t is linearly transformed to the jth position of the i-th layer, is the deviation parameter when the tree structure numbered t is linearly transformed to the jth position of the i-th layer.

[0112] In one possible implementation, represents the score (LeafScore) of the jth leaf node of the tth tree. The scores of the multiple leaf nodes each satisfy the following formula (6):

[0113]

[0114] Among them, L t represents the number of leaf nodes in the tree numbered t, represents the data of the jth leaf node in the tree numbered t, g j =-log(-log(u j )),u j ~U(0,1) is the Gumbel distribution. The smaller τ is (τ→0), the sampled vector tends to be a one-hot vector; the larger τ is (τ→∞), the sampled vector tends to be uniformly distributed.

[0115] It should be understood that the Gumbel-Softmax function can make the calculated leaf node score In the form of a one-hot vector, that is, a vector with only one element being 1 and the rest being 0, the leaf node with the highest score can be selected as the weight parameter C corresponding to the target leaf node by using the following formula (7): t and the deviation parameter d t .

[0116]

[0117] in, and Leaf nodes The corresponding weight parameters and bias parameters, is the leaf node score calculated by the above formula (6).

[0118] It should be understood that the weight parameters and bias parameters involved in the embodiments of the present application are obtained after training the deep tree intersection model, and the training process is the conventional training process for the deep learning model in the prior art, which will not be repeated here.

[0119] The deep tree intersection model provided in this embodiment includes multiple tree structures, each of which can calculate a target leaf node. The weight parameters and deviation parameters of the target leaf node obtained according to the data to be processed are a set of target physical parameters selected for each tree structure.

[0120] Figure 7 shows a two-dimensional deep tree intersection model, such as Figure 7 As shown, the model includes 2 tree structures, each tree structure includes 4 leaf nodes, that is, each tree has 4 sets of physical parameters, and the physical parameters of the 8 leaf nodes of the 2 numbers can be composed as follows Figure 5 The 16 logic parameters are shown.

[0121] The above step 205 can, for example, determine the current weight of the recurrent neural network model based on the weight parameter and bias parameter corresponding to the target leaf node of each tree structure. The specific implementation method is as follows:

[0122] The weight parameters corresponding to the target leaf nodes of each tree structure are summed or multiplied to obtain a first logic parameter; the deviation parameters corresponding to the target leaf nodes of each tree structure are summed or multiplied to obtain a second logic parameter; according to the first logic parameter and the second logic parameter, the target parameter corresponding to the target leaf node is determined, and the target parameter is used as the current weight corresponding to the recurrent neural network model at the current moment.

[0123] The first logic parameter and the second logic parameter are target logic parameters selected according to the data to be processed. Figure 7 As shown, although the two-dimensional deep tree intersection model can have 16 combinations of logic parameters, the logic parameter combinations selected by tree 1 and tree 2 are 5 logic parameters, that is, only one set needs to be selected for the final result operation.

[0124] It should be understood that Figure 7 The two-dimensional deep tree intersection model shown is only exemplary, and the present application does not specifically limit the number of tree structures involved in the deep tree intersection model.

[0125] In a possible implementation, the first logic parameter A and the second logic parameter b satisfy the following formula (8):

[0126]

[0127] Among them, T is the number of tree structures in the deep tree comparison model.

[0128] In the deep tree intersection model for classification tasks, the processing result y obtained by the processed data x based on the first logical parameter A and the second logical parameter b satisfies the probability distribution obtained by the following formula (9).

[0129] y=softmax(A·x+b) (9)

[0130] S206: Process the data to be processed using the hidden layer corresponding to the current moment of the recurrent neural network model.

[0131] S207: According to the historical weight and the current weight, the data to be processed is corrected to obtain a processing result of the natural language processing task.

[0132] Among them, the job of the hidden layer is to convert the input data into data that can be used by the output layer. That is, the hidden layer can abstract the features of the input data into another dimensional space to show its more abstract features, which can be better linearly divided.

[0133] After the hidden layer processes the data to be processed, the processed data is corrected according to the historical weights obtained through network training and the current weights determined by the deep tree-banana model, so that the output results of the recurrent neural network model are more accurate.

[0134] The processing method of the natural language processing task provided in this embodiment replaces the current weights in the recurrent neural network model with the output results of the deep tree intersection model, thereby making the output results of the recurrent neural network model more accurate. At the same time, since the above method does not require every physical parameter of the leaf node to participate in the result calculation, it can save equipment computing power, reduce the resource requirements involved in the calculation during the processing of the data to be processed, and also improve the processing efficiency of the natural language processing task.

[0135] Figure 8 Schematic diagram of the process of processing a natural language processing task provided in the embodiment of the present application Figure 3 The deep tree intersection model in this embodiment is used to determine the historical weights of the historical input data of the recurrent neural network model. Figure 8 As shown, the processing method of the natural language processing task provided by this application includes:

[0136] S301: Obtaining data to be processed for a natural language processing task.

[0137] S302: Obtain the current weight of the recurrent neural network model, where the current weight corresponding to the recurrent neural network model at any time is the same.

[0138] S303: Determine the historical weight corresponding to the current moment of the recurrent neural network model according to the deep tree intersection model.

[0139] S304: Process the data to be processed using the hidden layer corresponding to the current moment of the recurrent neural network model.

[0140] S305: According to the historical weight and the current weight, the data to be processed is corrected to obtain a processing result of the natural language processing task.

[0141] Fig. 9 The structure of the recurrent neural network model provided in this embodiment is shown in FIG. Figure 2 .like Fig. 9 As shown, the historical weight W in the recurrent neural network model will be determined by the output of the deep tree intersection model.

[0142] parameter Figure 4 and Fig. 9 It can be seen that the current weight V corresponding to any time is the same, and the value of the current weight V is fixed. The current weight V can be obtained according to network training, for example.

[0143] Since the deep tree intersection model in this embodiment is used to determine the historical weight W, this step only needs to obtain the current weight V pre-set in the recurrent neural network model.

[0144] The specific implementation method of this embodiment is the same as that of the above Figure 4 The specific implementation of the method shown in the embodiment is similar. The difference between the two is: Figure 4 In the embodiment, the output result of the deep tree intersection model is used to replace the current weight V in the recurrent neural network model, while in this embodiment, the output result of the deep tree intersection model is used to replace the historical weight W in the recurrent neural network model. Therefore, the specific implementation method of this embodiment can be found in the above Figure 4 The explanation of the embodiments will not be repeated here.

[0145] The processing method for natural language processing tasks provided in this embodiment replaces the historical weights in the recurrent neural network model with the output results of the deep tree intersection model, thereby making the output results of the recurrent neural network model more accurate. At the same time, since the above method does not require every physical parameter of the leaf node to participate in the result calculation, it can save equipment computing power, reduce the resource requirements involved in the calculation during the processing of the data to be processed, and also improve the processing efficiency of the natural language processing tasks.

[0146] Fig.10 Schematic diagram of the structure of the processing device for natural language processing tasks provided by this application. Fig.10 As shown, the processing device 300 for the natural language processing task includes:

[0147] An acquisition module 301 is used to acquire data to be processed for a natural language processing task;

[0148] A processing module 302 is used to input the data to be processed into a recurrent neural network model to obtain a processing result output by the recurrent neural network model;

[0149] Among them, a pre-trained deep tree intersection model is deployed in the recurrent neural network model, and the deep tree intersection model is used to determine the historical weights of the historical input data of the recurrent neural network model or determine the current weights of the current input data of the recurrent neural network model.

[0150] Optionally, if the deep tree intersection model is used to determine the historical weights of the historical input data of the recurrent neural network model, the apparatus further includes: a determination module 303;

[0151] The acquisition module 301 is further used to acquire the current weight of the recurrent neural network model, and the current weight corresponding to the recurrent neural network model at any time is the same;

[0152] The determination module 303 is used to determine the historical weight corresponding to the current moment of the recurrent neural network model according to the deep tree intersection model;

[0153] The processing module 302 is used to use the hidden layer corresponding to the current moment of the recurrent neural network model to process the data to be processed, and to correct the data to be processed according to the historical weight and the current weight to obtain the processing result of the natural language processing task.

[0154] Optionally, if the deep tree intersection model is used to determine the current weight of the current input data of the recurrent neural network model, the device further includes: a determination module 303;

[0155] The acquisition module 301 is further used to acquire the historical weights of the recurrent neural network model, and the historical weights corresponding to the recurrent neural network model at any time are the same;

[0156] The determination module 303 is used to determine the current weight corresponding to the recurrent neural network model at the current moment according to the deep tree intersection model;

[0157] The processing module 302 is used to use the hidden layer corresponding to the current moment of the recurrent neural network model to process the data to be processed, and to correct the data to be processed according to the historical weight and the current weight to obtain the processing result of the natural language processing task.

[0158] Optionally, the processing module 302 is used to input the data to be processed into the input node of each tree structure in the deep tree intersection model to obtain multiple leaf node scores of each tree structure; based on the multiple leaf node scores of each tree structure, select the leaf node with the highest leaf node score of each tree structure as the target leaf node; and use the parameters corresponding to the target leaf node as the current weight corresponding to the recurrent neural network model at the current moment.

[0159] Optionally, the processing module 302 is used to input the data to be processed into the input node of each tree structure, and obtain multiple leaf node scores of each tree structure based on linear transformation and Gumbel distribution function.

[0160] Optionally, the processing module 302 is used to perform a sum or product operation on the weight parameters corresponding to the target leaf nodes of each tree structure to obtain a first logical parameter;

[0161] Performing a summation or product operation on the deviation parameters corresponding to the target leaf nodes of each tree structure to obtain a second logic parameter;

[0162] The determination module 303 is used to determine the target parameter corresponding to the target leaf node according to the first logical parameter and the second logical parameter, and use the target parameter as the current weight corresponding to the recurrent neural network model at the current moment.

[0163] Fig.11 A schematic diagram of the structure of the processing device for the natural language processing task provided in this application. Fig.11 As shown, the present application provides a processing device for a natural language processing task, and the processing device 400 for the natural language processing task includes: a receiver 401, a transmitter 402, a processor 403 and a memory 404.

[0164] Receiver 401, used for receiving instructions and data;

[0165] A transmitter 402, used for sending instructions and data;

[0166] Memory 404, for storing computer-executable instructions;

[0167] The processor 403 is used to execute the computer-executable instructions stored in the memory 404 to implement the various steps performed by the method for processing a natural language processing task in the above embodiment. For details, please refer to the relevant description in the embodiment of the method for processing a natural language processing task.

[0168] Optionally, the memory 404 may be independent or integrated with the processor 403 .

[0169] When the memory 404 is independently provided, the electronic device further includes a bus for connecting the memory 404 and the processor 403 .

[0170] The present application also provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, a processing method for a natural language processing task as performed by a processing device for the above-mentioned natural language processing task is implemented.

[0171] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0172] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0173] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for processing a natural language processing task, characterized in that: include: Obtain data to be processed for natural language processing tasks; Inputting the data to be processed into a recurrent neural network model to obtain a processing result output by the recurrent neural network model; Among them, a pre-trained deep tree intersection model is deployed in the recurrent neural network model, and the deep tree intersection model is used to determine the historical weights of the historical input data of the recurrent neural network model or determine the current weights of the current input data of the recurrent neural network model.

2. The method according to claim 1, characterized in that If the deep tree intersection model is used to determine the historical weights of the historical input data of the recurrent neural network model, then the data to be processed is input into the recurrent neural network model to obtain the processing result output by the recurrent neural network model, including: Obtaining the current weight of the recurrent neural network model, the current weight corresponding to the recurrent neural network model at any time being the same; Determine the historical weight corresponding to the current moment of the recurrent neural network model according to the deep tree intersection model; The hidden layer corresponding to the current moment of the recurrent neural network model is used to process the data to be processed, and the data to be processed is corrected according to the historical weight and the current weight to obtain the processing result of the natural language processing task.

3. The method according to claim 1, characterized in that If the deep tree intersection model is used to determine the current weight of the current input data of the recurrent neural network model, then the data to be processed is input into the recurrent neural network model to obtain the processing result output by the recurrent neural network model, including: Obtaining the historical weights of the recurrent neural network model, the historical weights corresponding to the recurrent neural network model at any time being the same; Determine the current weight of the recurrent neural network model at the current moment according to the deep tree intersection model; The hidden layer corresponding to the current moment of the recurrent neural network model is used to process the data to be processed, and the data to be processed is corrected according to the historical weight and the current weight to obtain the processing result of the natural language processing task.

4. The method according to claim 3, characterized in that Determining the current weight corresponding to the recurrent neural network model at the current moment according to the deep tree intersection model includes: Inputting the data to be processed into the input node of each tree structure in the deep tree intersection model to obtain multiple leaf node scores of each tree structure; Based on the scores of multiple leaf nodes of each tree structure, select the leaf node with the highest score of each leaf node of each tree structure as the target leaf node; The parameters corresponding to the target leaf node are used as the current weights corresponding to the recurrent neural network model at the current moment.

5. The method according to claim 4, characterized in that The step of inputting the data to be processed into the input node of each tree structure in the deep tree intersection model to obtain multiple leaf node scores of each tree structure includes: The data to be processed is input into the input node of each tree structure, and based on linear transformation and Gumbel distribution function, multiple leaf node scores of each tree structure are obtained.

6. The method according to claim 4, characterized in that The parameters corresponding to the target leaf nodes of each tree structure include weight parameters and bias parameters; the parameters corresponding to the target leaf nodes are used as the current weights corresponding to the recurrent neural network model at the current moment, including: Performing a sum or product operation on the weight parameters corresponding to the target leaf nodes of each tree structure to obtain a first logical parameter; Performing a summation or product operation on the deviation parameters corresponding to the target leaf nodes of each tree structure to obtain a second logic parameter; According to the first logical parameter and the second logical parameter, a target parameter corresponding to the target leaf node is determined, and the target parameter is used as the current weight corresponding to the recurrent neural network model at the current moment.

7. A processing device for a natural language processing task, characterized in that: The device comprises: An acquisition module is used to obtain the data to be processed for the natural language processing task; A processing module, used for inputting the data to be processed into a recurrent neural network model to obtain a processing result output by the recurrent neural network model; Among them, a pre-trained deep tree intersection model is deployed in the recurrent neural network model, and the deep tree intersection model is used to determine the historical weights of the historical input data of the recurrent neural network model or determine the current weights of the current input data of the recurrent neural network model.

8. The device according to claim 7, characterized in that The device further comprises: a determination module; The acquisition module is further used to acquire the current weight of the recurrent neural network model, and the current weight corresponding to the recurrent neural network model at any time is the same; The determination module is used to determine the historical weight corresponding to the current moment of the recurrent neural network model according to the deep tree intersection model; The processing module is used to use the hidden layer corresponding to the current moment of the recurrent neural network model to process the data to be processed, and to correct the data to be processed according to the historical weight and the current weight to obtain the processing result of the natural language processing task.

9. A processing device for a natural language processing task, characterized in that: include: Memory; processor; Wherein, the memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the processing method of the natural language processing task as described in any one of claims 1-6.

10. A computer storage medium, characterized in that: The computer storage medium stores computer execution instructions, which, when executed by a processor, are used to implement the processing method for a natural language processing task as described in any one of claims 1 to 6.

Images