Question generation method and device, and computer readable storage medium

Abstract

The application provides a question generation method and device and a computer readable storage medium. The method comprises: obtaining a first input, the first input containing an answer, an entity word in the answer and first indication information, the value of the first indication information indicating that the entity word appears in the answer or in a question corresponding to the answer; inputting the first input into a neural network model; and the neural network model generating a question corresponding to the answer according to the first input. The technical scheme of the embodiment of the application can control the model to generate different types of questions according to different indication information, and can improve the controllability of question generation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of natural language processing, and more particularly, to a question generation method and device and a computer readable storage medium. BACKGROUND

[0002] There are massive amounts of data on the Internet at present, and it is of great significance to organize these data into question and answer pairs, but it is very difficult to organize in some cases, for example, the organization of question and answer pairs existing in massive documents.

[0003] In order to realize the automatic organization of massive documents, the prior art applies an end-to-end neural network model to the generation of question and answer pairs, which achieves good results. For example, massive data on the Internet is input into a neural network model as an answer, and the neural network model will generate a corresponding question.

[0004] As is known, there can be multiple ways of asking questions for the same answer. In other words, different types of questions can be generated for the same answer. However, in the conventional technology, the type of question output by the neural network model cannot be controlled. SUMMARY

[0005] The present application provides a question generation method, device and computer readable storage medium, which can improve the controllability of question generation.

[0006] In a first aspect, a question generation method is provided, which comprises: obtaining a first input containing an answer, an entity word in the answer, and first indication information, the value of the first indication information indicating that the entity word appears in the answer or in a question corresponding to the answer; inputting the first input into a neural network model; and the neural network model generating a question corresponding to the answer according to the first input.

[0007] The value of the first indication information is used to indicate whether the entity word in the answer appears in the answer or in the question, which can control the model to generate different types of questions according to different indication information, thereby improving the controllability of question generation.

[0008] In combination with the first aspect, in some implementations of the first aspect, the neural network model is pre-trained according to first sample data, the first sample data including a sample answer, a sample question, an entity word in the sample answer, and sample first indication information.

[0009] In combination with the first aspect, in some implementations of the first aspect, the first input further contains second indication information, the second indication information being used to indicate the category of the entity word.

[0010] According to the second indication information, a similar question can be generated by referring to the same type of entity word in the neural network model, thereby enhancing the migration ability of the neural network model.

[0011] In some implementations of the first aspect, the neural network model is pre-trained according to second sample data, the second sample data including a sample answer, a sample question, an entity word in the sample answer, sample first indication information, and sample second indication information.

[0012] In some implementations of the first aspect, before the first input is obtained, the method further includes randomly generating a value of the first indication information.

[0013] Randomly generating the value of the first indication information can improve the diversity of the question output by the neural network model.

[0014] In some implementations of the first aspect, the neural network model is an encoder-decoder model with an attention mechanism.

[0015] The attention mechanism is added to the encoder and the decoder of the neural network model to improve the generation effect of the model.

[0016] In some implementations of the first aspect, the encoder or the decoder uses a recurrent neural network (RNN) or a bi-directional long short-term memory (bi-LSTM).

[0017] Optionally, the encoder and the decoder can both use an RNN, or both use a bi-LSTM, or use different networks, for example, the encoder uses an RNN and the decoder uses a bi-LSTM.

[0018] In some implementations of the first aspect, the first indication information indicates that the entity word appears in the answer or appears in a subject word of a question corresponding to the answer.

[0019] The second aspect provides a question generation device, which includes: an obtaining module configured to obtain a first input, the input including an answer, an entity word in the answer, and a value of first indication information, the first indication information indicating that the entity word appears in the answer or appears in a question corresponding to the answer; an input module configured to input the first input into a neural network model; and a first generation module configured to generate, by the neural network model, a question corresponding to the answer according to the first input.

[0020] The value of the first indication information is used to indicate whether the entity word in the answer appears in the answer or in the question, different types of questions can be generated according to different indication information, and controllability of question generation is improved.

[0021] With reference to the second aspect, in some implementations of the second aspect, the apparatus further includes a first training module configured to pre-train the neural network model according to first sample data, the first sample data including a sample answer, a sample question, an entity word in the sample answer, and sample first indication information.

[0022] With reference to the second aspect, in some implementations of the second aspect, the apparatus further includes a second generation module configured to generate second indication information, the second indication information being used to indicate a category of the entity word.

[0023] According to the second indication information, the entity word is classified, and a similar question can be generated by referring to the same type of entity word that has been seen in the neural network model, so that the migration ability of the neural network model can be enhanced.

[0024] With reference to the second aspect, in some implementations of the second aspect, the apparatus further includes a second training module configured to pre-train the neural network model according to second sample data, the second sample data including a sample answer, a sample question, an entity word in the sample answer, sample first indication information, and sample second indication information.

[0025] With reference to the second aspect, in some implementations of the second aspect, the apparatus further includes a third generation module configured to randomly generate a value of the first indication information.

[0026] The first indication information is randomly generated, and the diversity of the question output by the neural network model can be improved.

[0027] With reference to the second aspect, in some implementations of the second aspect, the neural network model is an encoder-decoder model with an attention mechanism.

[0028] In the embodiments of the present application, an attention mechanism is added between the encoder and the decoder of the neural network model to improve the generation effect of the model.

[0029] With reference to the second aspect, in some implementations of the second aspect, the encoder or the decoder adopts RNN or bi-LSTM.

[0030] Optionally, the encoder and the decoder can adopt RNN at the same time, or can adopt bi-LSTM at the same time, or can adopt different networks, for example, the encoder adopts RNN and the decoder adopts bi-LSTM.

[0031] With reference to the second aspect, in some implementations of the second aspect, the first indication information indicates that the entity word appears in the answer or appears in a subject word of a question corresponding to the answer.

[0032] In a third aspect, a computer-readable storage medium is provided, including a computer program, when running on a computer device, causing a processing unit in the computer device to execute the method according to the first aspect.

[0033] In a fourth aspect, a computer program product is provided, including a computer program, when running on a computer device, causing a processing unit in the computer device to execute the method according to the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 is an application scenario example diagram of natural language processing provided by an embodiment of the present application;

[0035] Figure 2 is another application scenario example diagram of natural language processing provided by an embodiment of the present application;

[0036] Figure 3 is an example diagram of a related device of natural language processing provided by an embodiment of the present application;

[0037] Figure 4 is an example diagram of a system architecture provided by an embodiment of the present application;

[0038] Figure 5 is an example diagram of generating a question according to a CNN model provided by an embodiment of the present application;

[0039] Figure 6 is another example diagram of generating a question according to a CNN model provided by an embodiment of the present application;

[0040] Figure 7 is an example diagram of a hardware structure of a chip provided by an embodiment of the present application;

[0041] Figure 8 is an example diagram of a question generation method provided by an embodiment of the present application;

[0042] Figure 9 is an example diagram of generating a question by a neural network model provided by an embodiment of the present application;

[0043] Figure 10 is an example diagram of a method for training a neural network model according to an embodiment of the present application;

[0044] Figure 11 is an example diagram of a question generation device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments of the present application.

[0046] The embodiments of the present application can be used to convert answers into questions. The embodiments of the present application do not make specific limitations on the source of the answers, which can be derived from Internet data, that is, can be used to organize Internet data to obtain question and answer pairs. The Internet data can be data in manuals, internal materials, web pages and other documents on the Internet.

[0047] For ease of understanding, first, the related background art of the embodiments of the present application is briefly introduced.

[0048] In order to better utilize the massive data on the current Internet, it is of great significance to organize these data into question and answer pairs. The traditional technology usually generates question and answer pairs by using an end-to-end model, which shows good results on many challenging data sets. For example, the neural network model in the end-to-end model can use the massive data on the Internet as answers to generate questions. Exemplarily, the neural network model using the encoder-decoder architecture, the generation mode of the question and answer pair is mainly to input the answer and the entity word into the neural network model, and the model outputs the question. The intermediate loop layer can use RNN or bi-LSTM, or other neural network models such as convolutional neural networks (CNN), and the model architecture can also introduce an attention mechanism. In addition, other methods will further filter the input content, locate to the segment or entity that is more meaningful to generate questions, and add the original paragraph as the input of the model.

[0049] As known, for the same answer, there can be multiple ways of asking questions. In other words, for the same answer, different types of questions can be generated. For example, for the answer "the capital of Henan Province is Zhengzhou", the question can have two forms: "where is the capital of Henan Province" and "which province is Zhengzhou the capital of". The first form takes "Henan" as the theme word of the question, and the second form takes "Henan" as the answer. However, the traditional technology does not consider the semantic role of the entity in the input paragraph in the answer or the question, but mechanically takes these entities and answers as inputs, so the generated question is fixed.

[0050] The embodiments of the present application use the value of the first indication information to indicate whether the entity word in the answer appears in the answer or the question, and can control the model to generate different types of questions according to different indication information, thereby improving the controllability of question generation.

[0051] In order to better understand the scheme of the embodiments of the present application, first, the scheme of the embodiments of the present application will be described in conjunction withFigures 1 to 3 A simple introduction is given to possible application scenarios of the embodiments of the present application.

[0052] Figure 1 A natural language processing system is shown, which includes a user device and a data processing device. The user device includes a mobile phone, a personal computer, or an intelligent terminal such as an information processing center. The user device is the initiator of natural language data processing and the initiator of language query or inquiry, and usually the user initiates the request through the user device.

[0053] The data processing device can be a cloud server, a network server, an application server, a management server, or other devices or servers with data processing functions. The data processing device receives query statements / voices / texts from the intelligent terminal through an interactive interface, and then performs language data processing in the form of machine learning, deep learning, searching, reasoning, decision-making, etc. through the storage memory and the data processing processor. The storage memory in the data processing device can be a general term, including local storage and a database storing historical data. The database can be on the data processing device or on other network servers.

[0054] In Figure 1 In the natural language processing system shown, the user device can receive the user's instructions, input the answer, the entity words in the answer, and the first indication information into the user device, and then send a request to the data processing device, so that the data processing device analyzes the answer, the entity words in the answer, and the first indication information input by the user, and then outputs the question corresponding to the answer.

[0055] In Figure 1 The data processing device can execute the question generation method of the embodiments of the present application.

[0056] Figure 2 Another natural language processing system is shown, in Figure 2 In the natural language processing system shown, the user device can directly receive input from the user and directly process the input by the hardware of the user device itself. The specific process is similar to Figure 1 and can refer to the above description, which will not be repeated here.

[0057] In Figure 2 In the natural language processing system shown, the user device can receive the user's instructions, input the answer, the entity words in the answer, and the first indication information into the user device, and then analyze the answer, the entity words in the answer, and the first indication information input by the user by the user device itself, and then output the question corresponding to the answer.

[0058] In Figure 2In this way, the user equipment itself can perform the problem generation method of the embodiments of the present application.

[0059] Figure 3 is a schematic diagram of a related device for natural language processing provided by the embodiments of the present application.

[0060] The above Figure 1 and Figure 2 The user equipment in the above Figure 3 The local device 301 or the local device 302 in the above Figure 1 The data processing device in the above Figure 3 The execution device 210 in the above, wherein the data storage system 250 can store the to-be-processed data of the execution device 210, and the data storage system 250 can be integrated on the execution device 210, or can be arranged on a cloud or other network server.

[0061] Figure 1 and Figure 2 The processor in the above can perform data training / machine learning / depth learning through a neural network model or other models (for example, a model based on a support vector machine), and finally obtain a model through data training or learning, and take the answer, the entity words in the answer and the first indication information as inputs, and generate a corresponding question.

[0062] Since the embodiments of the present application involve the application of a large number of neural networks, in order to facilitate understanding, the related terms and concepts of the neural networks that may be involved in the embodiments of the present application will be introduced first.

[0063] (1) Neural network

[0064] The neural network can be composed of neural units, and the neural unit can refer to an operation unit with x s and intercept 1 as input, and the output of the operation unit can be:

[0065]

[0066] wherein s = 1, 2, … n, n is a natural number greater than 1, W s is x swhere b is the bias of the neuron. f is the activation function of the neuron, which is used to introduce non-linear characteristics into the neural network to convert the input signal in the neuron to an output signal. The output signal of the activation function can be used as the input of the next convolutional layer, and the activation function can be a sigmoid function. The neural network is a network formed by connecting a plurality of the above single neurons, that is, the output of one neuron can be the input of another neuron. The input of each neuron can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, and the local receptive field can be a region composed of a plurality of neurons.

[0067] (2) Deep neural network

[0068] A deep neural network (DNN), also known as a multi-layer neural network, can be understood as a neural network with multiple hidden layers. According to the position of different layers, the neural network inside the DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally, the first layer is the input layer, the last layer is the output layer, and the number of layers in between is the hidden layer. The layers are fully connected, that is, any neuron in the i-th layer is connected to any neuron in the i+1-th layer.

[0069] Although the DNN looks very complex, it is actually not complex in terms of the work of each layer. In simple terms, it is expressed as the following linear relationship expression: wherein, is the input vector, is the output vector, is the offset vector, W is the weight matrix (also called coefficient), and a() is the activation function. Each layer only performs the following simple operation on the input vector to obtain the output vector Due to the large number of layers in the DNN, the number of coefficients W and offset vectors is also relatively large. These parameters in the DNN are defined as follows: taking the coefficient W as an example: assuming that in a three-layer DNN, the linear coefficient of the fourth neuron in the second layer to the second neuron in the third layer is defined as The superscript 3 represents the layer number of the coefficient W, and the subscripts correspond to the output third layer index 2 and the input second layer index 4.

[0070] In summary, the coefficient of the k-th neuron in the L-1-th layer to the j-th neuron in the L-th layer is defined as

[0071] Note that the input layer has no W parameters. In deep neural networks, more hidden layers allow the network to better capture the complexity of real-world situations. In theory, the more parameters a model has, the higher its complexity, and the greater its "capacity" to learn more complex tasks. Training a deep neural network is the process of learning the weight matrices, and the ultimate goal is to obtain the weight matrices of all layers of the trained deep neural network (the weight matrices formed by the vectors W of many layers).

[0072] (3) Convolutional Neural Network

[0073] A convolutional neural network (CNN) is a deep neural network with a convolutional structure. The convolutional neural network includes a feature extractor composed of a convolutional layer and a subsampling layer, which can be regarded as a filter. The convolutional layer refers to the neuron layer in the convolutional neural network that performs convolution processing on the input signal. In the convolutional layer of the convolutional neural network, a neuron can be connected only to part of the adjacent layer neurons. In a convolutional layer, there are usually several feature planes, each of which can be composed of some rectangularly arranged neural units. The neural units in the same feature plane share weights, and the shared weights are the convolution kernel. The shared weights can be understood as being independent of the way and position of extracting image information. The convolution kernel can be initialized in the form of a matrix of random size, and the convolution kernel can obtain reasonable weights through learning in the training process of the convolutional neural network. In addition, the direct benefit of shared weights is to reduce the connections between layers of the convolutional neural network, while also reducing the risk of overfitting.

[0074] (4) Recurrent Neural Networks (RNN) are used to process sequence data. In traditional neural network models, it is from the input layer to the hidden layer and then to the output layer, and the layers are fully connected, while the nodes within each layer are not connected. Although this ordinary neural network has solved many difficult problems, it is still powerless for many problems. For example, to predict the next word of a sentence, the previous words need to be used because the words in a sentence are not independent. RNN is called recurrent neural network because the current output of a sequence is related to the previous output. The specific form is that the network will memorize the previous information and apply it to the calculation of the current output, that is, the nodes within the hidden layer are no longer unconnected but connected, and the input of the hidden layer includes not only the output of the input layer but also the output of the previous hidden layer. In theory, RNN can process sequence data of any length. The training of RNN is the same as the training of traditional CNN or DNN.

[0075] Since we already have convolutional neural networks (CNNs), why do we need recurrent neural networks (RNNs)? The reason is simple. CNNs rely on the fundamental assumption that elements are independent of each other, and that input and output are also independent—like a cat and a dog. However, in the real world, many elements are interconnected. For example, stock prices fluctuate over time. Or, imagine someone saying, "I love traveling, and my favorite place is Yunnan. I definitely want to go there someday." Humans know the answer to this question is "Yunnan." Humans can infer from context, but how can machines do the same? This is where RNNs come in. RNNs aim to give machines the ability to remember, just like humans. Therefore, the output of an RNN depends on both the current input information and historical memory information.

[0076] (5) Loss Function

[0077] In training a deep neural network, to ensure the output closely approximates the desired predicted value, we compare the network's prediction with the target value. Based on the difference, we update the weight vector of each layer (usually pre-configuring parameters before the initial update). For example, if the prediction is too high, the weight vector is adjusted to predict a lower value. This adjustment continues until the deep neural network predicts the target value or a value very close to it. Therefore, we need to predefine "how to compare the difference between the predicted and target values," which is the loss function or objective function. These are important equations used to measure the difference between the predicted and target values. Taking the loss function as an example, a higher output value (loss) indicates a greater difference, and training the deep neural network becomes a process of minimizing this loss.

[0078] (6) Backpropagation algorithm

[0079] Neural networks can employ backpropagation (BP) to correct the parameters of the initial neural network model during training, thereby reducing the reconstruction error loss. Specifically, forward propagation of the input signal to the output generates error loss; this error loss information is then propagated back to update the parameters of the initial neural network model, leading to convergence of the error loss. The backpropagation algorithm is an error-loss-driven backpropagation process aimed at obtaining the optimal parameters of the neural network model, such as the weight matrix.

[0080] like Figure 4 As shown, this application embodiment provides a system architecture 100. InFigure 4 In the embodiment, the data collection device 160 is configured to collect training data, and the training data in the embodiment includes sample answers, sample questions, entity words in the sample answers, and sample first indication information. After the training data is collected, the data collection device 160 stores the training data in the database 130, and the training device 120 trains the target model / rule 101 based on the training data maintained in the database 130.

[0081] The training device 120 trains the target model / rule 101 based on the training data, and the training device 120 processes the input training answers, entity words in the training answers, training first indication information, and training questions, compares the output questions with the training questions, and completes the training of the target model / rule 101 until the difference between the output questions and the training questions is less than a certain threshold.

[0082] The target model / rule 101 described above can be used to implement the question generation method in the embodiment, that is, the answer, the entity words in the answer, and the first indication information are input into the target model / rule 101 after being preprocessed (which can be processed by the preprocessing module 113 and / or the preprocessing module 114), and the corresponding question can be obtained. The target model / rule 101 in the embodiment can be a neural network. It should be noted that in actual application, the training data maintained in the database 130 can not all be collected by the data collection device 160, but can also be received from other devices. In addition, it should be noted that the training device 120 can not completely train the target model / rule 101 based on the training data maintained in the database 130, but can also obtain training data from the cloud or other places to train the model. The above description should not be regarded as a limitation of the embodiment.

[0083] The target model / rule 101 trained by the training device 120 can be applied to different systems or devices, such as the execution device 110 shown in the embodiment. Figure 4 The execution device 110 can be a terminal such as a mobile phone terminal, a tablet computer, a notebook computer, an augmented reality (AR) / virtual reality (VR), a vehicle-mounted terminal, and the like, and can also be a server or a cloud, etc. In the embodiment, the execution device 110 is configured to obtain the target model / rule 101 from the training device 120. Figure 4 In the embodiment, the execution device 110 is configured with an input / output (I / O) interface 112 for data interaction with external devices. The user can input data to the I / O interface 112 through the client device 140, and the input data in the embodiment can include answers input by the client device, entity words in the answers, and first indication information.

[0084] The preprocessing module 113 and the preprocessing module 114 are used to preprocess the input data received by the I / O interface 112. In this embodiment, the preprocessing module 113 and the preprocessing module 114 may be omitted (or only one of them may be used), and the calculation module 111 may be used directly to process the input data.

[0085] During the preprocessing of input data by the execution device 110, or during the calculation module 111 of the execution device 110 performing calculations and other related processes, the execution device 110 can call data, code, etc. in the data storage system 150 for corresponding processing, or store the data, instructions, etc. obtained from the corresponding processing into the data storage system 150.

[0086] Finally, I / O interface 112 feeds back the processing result, such as the question corresponding to the answer, to client device 140.

[0087] It is worth noting that the training device 120 can generate a target model / rule 101 for different downstream systems. The corresponding target model / rule 101 can be used to achieve the above objectives or complete the above tasks, thereby providing the user with the required results.

[0088] exist Figure 4 In the scenario shown, the user can manually provide input data (e.g., input a piece of text), which can be done through the interface provided by I / O interface 112. Alternatively, the client device 140 can automatically send input data (e.g., input a piece of text) to I / O interface 112. If user authorization is required for the client device 140 to automatically send input data, the user can set the corresponding permissions in the client device 140. The user can view the output results of the execution device 110 on the client device 140, which can be presented in various forms such as display, sound, or animation. The client device 140 can also act as a data acquisition terminal, collecting the input data and output results of the input I / O interface 112 as new sample data and storing them in the database 130. Alternatively, the data can be collected directly from the I / O interface 112 without going through the client device 140, using the input data and output results of the input I / O interface 112 as new sample data and storing them in the database 130.

[0089] It is worth noting that, Figure 4 This is merely a schematic diagram of a system architecture provided in an embodiment of this application, and the positional relationships between the devices, components, modules, etc., shown in the diagram do not constitute any limitation. For example, in Figure 4In this context, the data storage system 150 is an external memory relative to the execution device 110. In other cases, the data storage system 150 may also be placed within the execution device 110.

[0090] like Figure 4 As shown, the target model / rule 101 is obtained by training according to the training device 120. The target model / rule 101 can be the problem generation and detection model in the embodiment of this application. Specifically, the neural network provided in the embodiment of this application can be CNN, deep convolutional neural network (DCNN), recurrent neural network (RNN), etc.

[0091] Since CNN is a very common type of neural network, the following will combine... Figure 5 This section focuses on a detailed explanation of the structure of CNNs. As mentioned in the basic concept introduction above, a Convolutional Neural Network (CNN) is a deep neural network with a convolutional structure. It is a deep learning architecture, which refers to learning at multiple levels of abstraction through machine learning algorithms. As a deep learning architecture, CNN is a feed-forward artificial neural network, where each neuron responds to the input image.

[0092] like Figure 5 As shown, the Convolutional Neural Network (CNN) 200 may include an input layer 110, convolutional / pooling layers 120 (where pooling layers are optional), and neural network layers 130. The details of these layers are described below.

[0093] Convolutional / pooling layers 120:

[0094] Convolutional layers:

[0095] like Figure 5 The convolutional / pooling layer 120 shown may include layers 121-126 as in Examples 121-126. In one implementation, layer 121 is a convolutional layer, layer 122 is a pooling layer, layer 123 is a convolutional layer, layer 124 is a pooling layer, layer 125 is a convolutional layer, and layer 126 is a pooling layer. In another implementation, layers 121 and 122 are convolutional layers, layer 123 is a pooling layer, layers 124 and 125 are convolutional layers, and layer 126 is a pooling layer. That is, the output of the convolutional layer can be used as the input of a subsequent pooling layer, or as the input of another convolutional layer to continue the convolution operation.

[0096] Taking convolutional layer 121 as an example, convolutional layer 121 can include many convolution operators, also known as kernels. In natural language processing, its role is equivalent to a filter that extracts specific information from the input speech or semantic information. The convolution operator can essentially be a weight matrix, which is usually predefined.

[0097] The weight values ​​in these weight matrices need to be obtained through extensive training in practical applications. The weight matrices formed by the weight values ​​obtained through training can extract information from the input image, thereby helping the convolutional neural network 100 to make correct predictions.

[0098] When a convolutional neural network 100 has multiple convolutional layers, the initial convolutional layers (e.g., 121) tend to extract more general features, which can also be called low-level features. As the depth of the convolutional neural network 100 increases, the features extracted by later convolutional layers (e.g., 126) become more and more complex, such as high-level semantic features. Features with higher semantic levels are more suitable for the problem to be solved.

[0099] Pooling layer:

[0100] Because it is often necessary to reduce the number of training parameters, pooling layers are often introduced periodically after convolutional layers, i.e., ... Figure 5 In example 120, layers 121-126 can be a convolutional layer followed by a pooling layer, or multiple convolutional layers followed by one or more pooling layers. In natural language data processing, the sole purpose of pooling layers is to reduce the size of the data.

[0101] Neural network layer 130:

[0102] After processing by the convolutional / pooling layers 120, the convolutional neural network 100 is still insufficient to output the required information. As mentioned earlier, the convolutional / pooling layers 120 only extract features and reduce the parameters introduced by the input data. However, to generate the final output information (the required class information or other relevant information), the convolutional neural network 100 needs to utilize neural network layers 130 to generate one or more outputs representing the required number of classes. Therefore, neural network layers 130 may include multiple hidden layers (such as...). Figure 5 As shown in 131, 132 to 13n) and output layer 140, the parameters contained in these multi-layer hidden layers can be pre-trained based on relevant training data for specific task types, such as speech or semantic recognition, classification or generation, etc.

[0103] After the multiple hidden layers in neural network layer 130, the final layer of the entire convolutional neural network 100 is the output layer 140. This output layer 140 has a loss function similar to classification cross-entropy, specifically used to calculate the prediction error. Once the entire convolutional neural network 100 has undergone forward propagation (e.g., ...), the loss function is applied. Figure 5 The propagation from 110 to 140 is completed (forward propagation), and the reverse propagation (such as...) Figure 5 The propagation from 140 to 110 (backpropagation) will begin to update the weight values ​​and biases of the layers mentioned above, in order to reduce the loss of the convolutional neural network 100 and the error between the output of the convolutional neural network 100 through the output layer and the ideal result.

[0104] It should be noted that, as Figure 5 The convolutional neural network 100 shown is only an example of a convolutional neural network. In specific applications, convolutional neural networks can also exist in the form of other network models.

[0105] like Figure 6 As shown, the Convolutional Neural Network (CNN) 200 may include an input layer 110, convolutional / pooling layers 120 (where pooling layers are optional), and neural network layers 130. Figure 6 In the process, multiple convolutional / pooling layers in the convolutional / pooling layer 120 are parallel, and the extracted features are all input to the full neural network layer 130 for processing.

[0106] Figure 7 This is a schematic diagram of the hardware structure of a chip provided in an embodiment of this application. The chip includes a neural processing unit (NPU) 50. The chip can be configured as follows: Figure 4 The execution device 110 shown is used to perform the calculations of the calculation module 111. This chip can also be located in, for example... Figure 4 The training device 120 shown is used to complete the training work of the training device 120 and output the target model / rule 101. For example... Figure 5 and Figure 6 The algorithms for each layer in the convolutional neural network shown can all be implemented in, for example... Figure 7 This is achieved in the chip shown.

[0107] The question generation method of this application embodiment can be executed in the arithmetic circuit 503 and / or vector calculation unit 507 in the NPU 50 to obtain the question corresponding to the answer.

[0108] The following is a brief introduction to the various modules and units in the NPU 50.

[0109] The NPU 50, acting as a coprocessor, can be mounted on the host CPU, which then assigns tasks to it. The core of the NPU 50 is the arithmetic circuit 503. When the NPU 50 is working, the controller 504 within the NPU 50 can control the arithmetic circuit 503 to retrieve data from the memory (weight memory or input memory) and perform calculations.

[0110] In some implementations, the arithmetic circuit 503 internally includes multiple process engines (PEs). In some implementations, the arithmetic circuit 503 is a two-dimensional pulsating array. The arithmetic circuit 503 can also be a one-dimensional pulsating array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 503 is a general-purpose matrix processor.

[0111] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory 502 and caches it in each PE of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory 501 and performs matrix operations with matrix B. The partial result or the final result of the obtained matrix is ​​stored in the accumulator 508.

[0112] The vector computation unit 507 can further process the output of the computation circuit, such as vector multiplication, vector addition, exponentiation, logarithmic operations, size comparisons, etc. For example, the vector computation unit 507 can be used for network computation in non-convolutional / non-fully connected (FC) layers of neural networks, such as pooling, batch normalization, and local response normalization.

[0113] In some implementations, vector computation unit 507 can store the processed output vector into unified buffer 506. For example, vector computation unit 507 can apply a nonlinear function to the output of arithmetic circuit 503, such as a vector of accumulated values, to generate activation values. In some implementations, vector computation unit 507 generates normalized values, merged values, or both. In some implementations, the processed output vector can be used as activation input to arithmetic circuit 503, for example, for use in subsequent layers of a neural network.

[0114] The unified memory 506 is used to store input data and output data.

[0115] The weight data is directly transferred from the external memory to the input memory 501 and / or the unified memory 506 through the direct memory access controller 505 (DMAC), the weight data in the external memory is stored in the weight memory 502, and the data in the unified memory 506 is stored in the external memory.

[0116] The bus interface unit (BIU) 510 is used to enable interaction between the main CPU, DMAC and instruction fetch memory 509 via a bus.

[0117] The instruction fetch buffer 509, which is connected to the controller 504, is used to store the instructions used by the controller 504.

[0118] The controller 504 is used to call the instructions cached in the instruction memory 509 to control the operation of the computing accelerator.

[0119] Generally, the unified memory 506, input memory 501, weighted memory 502, and instruction fetch memory 509 can all be on-chip memories. The external memory of the NPU can be memory located outside the NPU, such as double data rate synchronous dynamic random access memory (DDR SDRAM), high bandwidth memory (HBM), or other readable and writable memory.

[0120] The problem generation method of this application embodiment will now be described in detail with reference to the accompanying drawings. Figure 1 Data processing equipment in China Figure 2 User equipment in Figure 3 The execution device 210 and Figure 4 The execution device 110 and other devices in the process are executed.

[0121] Figure 8 This is an example diagram of the problem generation method according to an embodiment of this application. For example... Figure 8 As shown, the method 800 includes steps S810, S820 and S830, which are described in detail below.

[0122] S810, obtain the first input.

[0123] S820, the first input is input into the neural network model.

[0124] As one embodiment, the first input includes an answer, entity words in the answer, and first indication information. The value of the first indication information indicates that the entity word appears in the answer or in the question corresponding to the answer.

[0125] As another embodiment, the first input may also include second indication information, which is used to indicate the category of the entity word.

[0126] It should be understood that entity words can be categorized according to their type, level, or nature, or they can be categorized according to their major or minor categories, such as writers, singers, cities, animals, plants, etc. The specific classification method is not limited here.

[0127] Classifying entity words based on the second indication information allows for the generation of similar questions by referencing similar entity words already seen in the neural network model, thereby enhancing the transferability of the neural network model.

[0128] It should be understood that the value of the first indication information is randomly generated before the first input is obtained. Optionally, the value can be manually selected according to expectations, or the programmer can control the probability of the value of the first indication information when writing code, thereby controlling the type of problem.

[0129] Optionally, the first indication information can take the values ​​0 and 1, and can be marked as follows: when the entity word is expected to appear in the question, the value is marked as 1; when the entity word is expected to appear in the answer, the value is marked as 0. It should be understood that the value of the first indication information can be 0 and 1, or other integers, or can be represented by specific symbols such as √ and ×, which are not limited here. For example, when the entity word "Henan" is expected to appear in the generated question, if the user manually selects to mark the value of the first indication information as 1 and uses it as input to the model, the output will be the question "What is the capital of Henan?", instead of generating the question "What is the capital of Zhengzhou?".

[0130] In another implementation, programmers can control the question type by controlling the probabilities of 0 and 1. For example, controlling the probabilities of 0 and 1 to be 20% and 80% respectively means that entity words are more likely to appear in the question. When the probabilities of 0 and 1 are 50% and 50% respectively, the probability of entity words appearing in the question and the answer is the same. When the probabilities of 0 and 1 are 80% and 20% respectively, the probability of entity words appearing in the answer is more likely.

[0131] Optionally, the first indication information may also indicate that the entity word appears in the answer or in the subject words of the question corresponding to the answer.

[0132] Optionally, entity words in the answer can be extracted from the answer in advance. Optionally, entity words can be extracted from the answer through a named entity recognition algorithm or through manual reading, without limitation.

[0133] S830, the neural network model generates the question corresponding to the answer based on the first input.

[0134] It should be understood that the neural network model is pre-trained based on sample data. In other words, the neural network model is pre-trained using sample data, and then the unknown answer to the question is input into the model to generate the corresponding question.

[0135] In this embodiment of the application, the neural network model can be as follows: Figure 9 The encoder-decoder architecture model is shown. Specifically, input A is encoded by the encoder, and input B is encoded simultaneously. Then, the results from the two encoders are combined into a single vector, which is used as the input to the decoder. The decoder then decodes the input and outputs the problem.

[0136] Wherein, input A is the answer, and optionally, input B can be the value of the entity word and the first instruction information in the answer, or it can be the value of the entity word, the first instruction information and the second instruction information in the answer.

[0137] Optionally, to improve model performance, an attention mechanism is added to the encoder and decoder in this embodiment of the application.

[0138] Figure 10 This is an example diagram of a method for training a neural network model according to an embodiment of this application. Figure 10 As shown, the method 1000 includes steps S1010, S1020 and S1030, which are described in detail below.

[0139] S1010, Extract entity words.

[0140] Entity words are words that can stand alone as sentence components, and can be: nouns, verbs, adjectives, numerals, quantifiers, and pronouns. In this embodiment, entity words in the answer are of great significance as input to the neural network model for question generation; therefore, entity words must be extracted from the samples before processing the sample data.

[0141] Optionally, entity words can be extracted from sample answers using a named entity recognition algorithm, or they can be selected from sample answers by manual reading.

[0142] S1020, Organize sample data.

[0143] As an example, the sample data may be first sample data including sample answers, sample questions, entity words in the sample answers, and sample first indication information.

[0144] Specifically, the value of the first indication information of the sample indicates that the entity word appears in the sample answer or in the sample question corresponding to the sample answer.

[0145] Optionally, the value of the first indication information of the sample is 0 and 1, and it can be marked as follows: when the entity word appears in the sample question, the value is marked as 1, and when the entity word appears in the sample answer, the value is marked as 0. It should be understood that the value of the first indication information of the sample can be 0 and 1, or it can be other integers, or it can be represented by specific symbols such as √ and ×, which is not limited here.

[0146] For example, for the sample answer "The capital of Henan is Zhengzhou," there are two sample questions: "What is the capital of Henan?" and "What province is Zhengzhou the capital of?". The entity word "Henan" is extracted. If the corresponding sample question is "What province is Zhengzhou the capital of?", it means the entity word "Henan" appears in the sample answer but not in the sample question. In this case, the value of the first indicator information is marked as 0. If the corresponding sample question is "What is the capital of Henan?", it means the entity word "Henan" appears in the sample question. In this case, the value of the first indicator information is marked as 1.

[0147] As another embodiment, the sample data may be second sample data including sample answers, sample questions, entity words in the sample answers, and sample first indication information, and also sample second indication information.

[0148] Specifically, the second indication information of the sample is used to indicate the category of the entity word. Optionally, the entity word can be categorized according to its type, level, or nature, or it can be categorized according to its major or minor categories.

[0149] Optionally, the category of entity words can also be extracted from the sample answers using a named entity recognition algorithm. It should be understood that the category of entity words can be extracted at the same time as the entity words are extracted.

[0150] Based on the second instruction information, entity words are categorized, which enhances the transferability of the neural network model. When encountering more unknown entities in the future, similar questions can be generated based on the similar entities that have been seen.

[0151] S1030, training the neural network model.

[0152] Optionally, a neural network model can be trained using sample answers, sample questions, entity words in the sample answers, and sample first indication information. Alternatively, a neural network model can be trained using sample answers, sample questions, entity words in the sample answers, sample first indication information, and sample second indication information.

[0153] Optionally, the neural network model can be an encoder-decoder model with an attention mechanism.

[0154] Figure 11 This is an example diagram of a problem generation apparatus according to an embodiment of this application. Figure 11 As shown, the device 1100 includes an acquisition module 1110, an input module 1120, and a first generation module 1130.

[0155] The acquisition module 1110 is used to acquire a first input, which includes an answer, entity words in the answer, and the value of a first indication information, wherein the first indication information indicates that the entity words appear in the answer or in the question corresponding to the answer; the input module 1120 is used to input the first input into a neural network model; and the first generation module 1130 is used for the neural network model to generate the question corresponding to the answer based on the first input.

[0156] Optionally, the device 1100 may further include a first training module for pre-training the neural network model based on first sample data, wherein the first sample data includes sample answers, sample questions, entity words in the sample answers, and sample first indication information.

[0157] Optionally, the device 1100 may further include a second generation module for generating second indication information, the second indication information being used to indicate the category of entity words.

[0158] Optionally, the device 1100 may further include a second training module for pre-training the neural network model based on second sample data, the second sample data including sample answers, sample questions, entity words in the sample answers, sample first indication information, and sample second indication information.

[0159] Optionally, the device 1100 may further include a third generation module for randomly generating the value of the first indication information.

[0160] Optionally, the device may further include an entity extraction module.

[0161] It should be understood that the neural network model may be an encoder-decoder model with an attention mechanism.

[0162] It should be understood that the encoder or the decoder may employ an RNN or a bi-LSTM.

[0163] It should also be understood that the first indication information indicates that the entity word appears in the answer or in the subject words of the question corresponding to the answer.

[0164] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0165] The methods in the embodiments of this application, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution or part of the technical solution of this application can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The storage medium includes at least: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, and other media capable of storing program code.

[0166] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A problem generation method, characterized in that, include: Obtain a first input, which includes an answer, entity words in the answer, and first indication information, wherein the value of the first indication information indicates that the entity words appear in the answer or in the question corresponding to the answer; The first input is fed into a neural network model, which is pre-trained based on the first sample data. The first sample data includes sample answers, sample questions, entity words in the sample answers, and sample first indication information. The neural network model generates the question corresponding to the answer based on the first input.

2. The method according to claim 1, characterized in that, The first input also includes second indication information, which is used to indicate the category of the entity word.

3. The method according to claim 2, characterized in that, The neural network model is pre-trained based on second sample data, which includes sample answers, sample questions, entity words in the sample answers, first sample indication information, and second sample indication information.

4. The method according to any one of claims 1-3, characterized in that, Prior to obtaining the first input, the method further includes: The value of the first indication information is randomly generated.

5. The method according to claim 4, characterized in that, The neural network model is an encoder-decoder model with an attention mechanism.

6. The method according to claim 5, characterized in that, The encoder or the decoder employs a recurrent neural network (RNN) or a bidirectional long short-term memory network (bi-LSTM).

7. The method according to any one of claims 1-3, characterized in that, The first indication information indicates that the entity word appears in the answer or in the subject words of the question corresponding to the answer.

8. A problem generation device, characterized in that, include: The acquisition module is used to acquire a first input, which includes an answer, entity words in the answer, and the value of a first indication information, wherein the first indication information indicates that the entity word appears in the answer or in the question corresponding to the answer; An input module is used to input the first input into the neural network model; The first training module is used to pre-train the neural network model based on the first sample data, wherein the first sample data includes sample answers, sample questions, entity words in the sample answers, and sample first indication information. The first generation module is used by the neural network model to generate the question corresponding to the answer based on the first input.

9. The apparatus according to claim 8, characterized in that, The device further includes: The second generation module is used to generate second indication information, which is used to indicate the category of entity words.

10. The apparatus according to claim 9, characterized in that, The device further includes: The second training module is used to pre-train the neural network model based on the second sample data, which includes sample answers, sample questions, entity words in the sample answers, sample first indication information, and sample second indication information.

11. The apparatus according to any one of claims 8-10, characterized in that, The device further includes: The third generation module is used to randomly generate the value of the first indication information.

12. The apparatus according to claim 11, characterized in that, The neural network model is an encoder-decoder model with an attention mechanism.

13. The apparatus according to claim 12, characterized in that, The encoder or the decoder employs a recurrent neural network (RNN) or a bidirectional long short-term memory network (bi-LSTM).

14. The apparatus according to any one of claims 8-10, characterized in that, The first indication information indicates that the entity word appears in the answer or in the subject words of the question corresponding to the answer.

15. A computer-readable storage medium, characterized in that, Includes a computer program that, when run on a computer device, causes a processing unit in the computer device to perform the method as described in any one of claims 1-7.

Images