Intelligent text reading comprehension method and device based on three-dimensional feature representation

Abstract

The application discloses a three-dimensional feature representation-based intelligent text reading comprehension method and device, a storage medium and an electronic device, and belongs to the fields of natural language processing and artificial intelligence; the technical problem to be solved by the application is how to capture direct interaction features between three-sequence data and how to enhance sufficient interaction among an article, a question and options, so as to improve the prediction accuracy of an intelligent text reading comprehension system; the technical scheme adopted is as follows: ① a three-dimensional feature representation-based intelligent text reading comprehension method, comprising the following modules: a pre-training embedding representation module, a feature filtering module, a 3D CNN interaction feature module and a label prediction module; ② a three-dimensional feature representation-based intelligent text reading comprehension device, comprising: an intelligent text reading comprehension data set acquisition unit, an intelligent text reading comprehension model construction unit and an intelligent text reading comprehension model training unit.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing, and specifically to an intelligent text reading comprehension method and apparatus based on three-dimensional feature representation. Background Technology

[0002] Machine reading comprehension refers to the task of finding the correct answers to related questions within a given text. It is one of the most popular research topics in natural language processing, and can be divided into four sub-tasks based on the way questions are answered: fill-in-the-blank reading comprehension, extractive reading comprehension, intelligent text reading comprehension, and generative reading comprehension. Intelligent text reading comprehension relies on a given text and requires selecting the correct answer from multiple options for a given question. The options are summarized and written by human experts and do not appear directly in the original text. Therefore, intelligent text reading comprehension requires computers to possess strong reasoning, analogy, and summarization abilities. This task not only needs to calculate the matching degree between the question and each candidate option but also the relevance between the question, options, and the text content. This is because some options may be grammatically correct and highly relevant to the question, but they may not fit the context of the text, thus they are not the correct answer. Therefore, intelligent text reading comprehension is a task that requires mining the interaction features among these three elements and calculating their relevance. To date, existing methods have not substantially solved this problem.

[0003] To achieve feature interaction between the text, questions, and options, most existing methods fuse text information into the questions or options in different ways, then perform interactive matching between the questions and options, and finally select the option with the highest match to the question as the correct answer. The advantage of this type of method is its simplicity and ease of implementation. However, the problem lies in the fact that intelligent text reading comprehension is essentially a task of determining the relationship between three text sequences (i.e., text, questions, and options). Utilizing text information through fusion, besides the information loss caused by the fusion process, inevitably leads to insufficient interaction between sequences, thus reducing the accuracy of the reading comprehension model.

[0004] To address the shortcomings of existing methods, this invention proposes an intelligent text reading comprehension method and apparatus based on three-dimensional feature representation. This method can capture the direct interaction features between three sequence data and effectively enhance the full interaction between the text, questions, and options, thereby improving the prediction accuracy of the intelligent text reading comprehension system. Summary of the Invention

[0005] The technical objective of this invention is to provide an intelligent text reading comprehension method, apparatus, storage medium, and electronic device based on three-dimensional feature representation, in order to address how to capture the direct interactive features between three sequence data and how to enhance the full interaction between the text, questions, and options, thereby improving the prediction accuracy of the intelligent text reading comprehension system.

[0006] The technical objective of this invention is achieved as follows: an intelligent text reading comprehension method based on three-dimensional feature representation, comprising the following steps:

[0007] S1. Obtain intelligent text reading comprehension datasets: Download publicly available intelligent text reading comprehension datasets from the internet or construct your own dataset;

[0008] S2. Constructing an intelligent text reading comprehension model: Constructing an intelligent text reading comprehension model based on three-dimensional feature representation;

[0009] S3. Train the intelligent text reading comprehension model: Train the intelligent text reading comprehension model constructed in step S2 on the intelligent text reading comprehension training dataset obtained in step S1.

[0010] Preferably, the specific steps for constructing the intelligent text reading comprehension model in step S2 are as follows:

[0011] S201, Constructing the Input Module

[0012] For each data point in the dataset, the text sequence is denoted as context; the question sequence as query; and all candidate options as response. Based on the correct answer, the label for each data point is determined: if the correct answer is A, it is denoted as 1000; if the correct answer is B, it is denoted as 0100; if the correct answer is C, it is denoted as 0010; and if the correct answer is D, it is denoted as 0001. These three text sequences and the label together constitute one input data point.

[0013] S202, Constructing pre-trained embedded representation modules

[0014] The pre-trained embedding representation module uses a pre-trained language model to perform embedding encoding operations on the input data constructed in S201, thereby obtaining the embedding representations of the text, questions, and options in the input data, denoted as follows: and The input constructed in S201 contains three text sequences. The article sequence uses a separate encoding module, while the question and option sequences share a common encoding module. Since the pre-trained language model itself contains a multi-layer encoding network, the output of different layers can be selected as its embedding representation based on different input objects. The specific implementation is shown in the following formula:

[0015]

[0016]

[0017]

[0018] In this equation (1.1), the pre-trained language model BERT is used to embed and encode the input article sequence context, and the subscript n represents the number of network layers within BERT. The text represents the embedding representation of the article; Formula (1.2) indicates the embedding encoding of the input question sequence query using the pre-trained language model BERT, where the subscript t indicates the number of network layers within BERT used. This represents the embedded representation of a question; formula (1.3) is basically the same as formula (1.2), except that the encoding object is the option response. This indicates an embedded representation of the options;

[0019] S203, Constructing the Feature Filtering Module

[0020] Embedded representations of articles, questions, and options using adaptive average pooling operations. and Feature filtering is performed to obtain the corresponding pooling feature representations, namely, the article pooling feature representation, the question pooling feature representation, and the option pooling feature representation, which are denoted as follows: and

[0021] S204. Constructing a 3D CNN interactive feature module

[0022] The module structure is as follows: Figure 5 As shown; its received feature filtering module outputs the article pooling feature representation Question Pooling Feature Representation and option pooling feature representation Then, two interactive feature construction methods are used to construct interactive features among the text, questions, and options, thereby obtaining two three-dimensional feature representations. and Finally, after processing by a 3DCNN structure, the final 3D interactive feature representation is obtained.

[0023] S205, Constructing a Label Prediction Module

[0024] The final 3D CNN interactive feature representation obtained in step S204 will be used as the input of this module. It will be processed by a fully connected network with a dimension of 4 and an activation function of softmax to obtain the probability of each candidate option as the correct answer. This module will predict the candidate option with the highest probability as the correct answer.

[0025] If the model has not yet been trained, step S3 needs to be executed to train it and optimize the model parameters; when the model has been trained, step S205 predicts which of the options is the correct answer.

[0026] Preferably, the specific steps for constructing the 3D CNN interactive feature module in step S204 are as follows:

[0027] S20401. Obtaining the 3D Representation 1: Using the Additive 3D Interactive Feature Construction Method to Represent the Article Pooling Features Question Pooling Feature Representation and option pooling feature representation The process is performed to obtain the three-dimensional feature representation 1, i.e. The specific implementation is shown in the following formula:

[0028]

[0029] Formula (2) represents the implementation through additive operations. and The interaction computation among the three, where i and j represent the input tensor. and The element index, I kl and J cl Representing the input tensors respectively and The number of elements in and Indicates the parameters to be trained;

[0030] S20402. Obtaining the 3D Representation 2: Using the Dot 3D Interactive Feature Construction Method to Represent the Article Pooling Features Question Pooling Feature Representation and option pooling feature representation The processing yields a three-dimensional feature representation 2, namely... The specific implementation is shown in the following formula:

[0031]

[0032] i = 1, 2, ..., I kl j = 1, 2, ..., J cl (3)

[0033] Formula (3) represents the implementation through dot product operation. and Interactive computation between the parties, where d represents the input tensor In the index of the element, D represents the input tensor. The number of elements in the input tensor, where i and j represent the number of elements in the input tensor. and The element index, I kl and J cl Representing the input tensors respectively and The number of elements in the tensor; Γ is a third-order tensor with shape D×D×D, where the element at position (d,d,d) has a value of 1 (d=1,2,…,D), and the element at other positions has a value of 0;

[0034] ×1, ×2, and ×3 represent modular multiplication (1, 2, and 3) respectively. These are custom tensor operators defined for easier formula writing and expression, and their operation process is as follows: Given two tensors... and but express and Modular multiplication of n, where Similarly, X and column vectors The modular multiplication rule of n is defined as in,

[0035] S20403, Obtaining 3D CNN Interactive Feature Representation 1: Using a 3D CNN to obtain the three-dimensional feature representation 1, i.e. Interaction feature extraction is performed to obtain the 3D CNN interaction feature representation 1, i.e. The specific implementation is shown in the following formula:

[0036]

[0037]

[0038]

[0039]

[0040] Formula (4.1) indicates that the f-th convolutional kernel represents 1 for the three-dimensional features, i.e. The result of convolving a specific region and mapping it using the ReLU function, where [x1, y1, z1] represents the size of the convolution kernel. This represents the weight matrix of the f-th convolutional kernel, where i, j, and k represent the x-coordinate, y-coordinate, and depth coordinates of the convolutional region, respectively, and m... l mh and m d The three-dimensional feature representation is 1, that is... The length, height, and depth of the region, i: i+x1-1, j: j+y1-1, k: k+z1-1, represent the convolution region. This represents the bias matrix of the f-th convolutional kernel. Let represent the convolution result of the f-th convolutional kernel in the regions i:i+x1-1, j:j+y1-1, k:k+z1-1; Formula (4.2) represents the integration of the horizontal and vertical convolution results of the f-th convolutional kernel in each region to obtain the k-th depth convolution result of the f-th convolutional kernel, where s x1 and s y1 Indicates the horizontal convolution stride and the vertical convolution stride. Let represent the depthwise convolution result of the f-th convolution kernel; Equation (4.3) represents integrating all depthwise convolution results of the f-th convolution kernel to obtain the depthwise convolution result of the f-th convolution kernel, where s z1 Indicates the stride of the depthwise convolution. Let f represent the depthwise convolution result of the f-th convolutional kernel; Equation (4.4) represents the integration of the depthwise convolution results of all convolutional kernels to obtain the 3D feature representation of this network layer, i.e. The final convolution result, where, This indicates that the convolutional neural network represents 1 for three-dimensional features, i.e. The final convolution result is called the 3D CNN interactive feature representation.

[0041] S20404, Obtaining 3D CNN Interactive Feature Representation 2: Using another 3D CNN for 3D feature representation 2, i.e. Interaction feature extraction is performed to obtain the 3D CNN interaction feature representation 2, i.e. The specific implementation is shown in the following formula:

[0042]

[0043]

[0044]

[0045]

[0046] Formula (5.1) indicates that the f-th convolutional kernel represents 2 of the three-dimensional features, i.e. The result of convolving a specific region and mapping it using the ReLU function, where [x2, y2, z2] represents the size of the convolution kernel. This represents the weight matrix of the f-th convolutional kernel, where i, j, and k represent the x-coordinate, y-coordinate, and depth coordinates of the convolutional region, respectively, and r... lr h and r d The three-dimensional feature representation is 2, that is The length, height, and depth of the region, i: i+x²-1, j: j+y²-1, and k: k+z²-1 represent the convolution region. This represents the bias matrix of the f-th convolutional kernel. Let f represent the convolution result of the f-th convolutional kernel in the regions i:i+x²-1, j:j+y²-1, and k:k+z²-1; Equation (5.2) represents the integration of the horizontal and vertical convolution results of the f-th convolutional kernel in each region to obtain the k-th depth convolution result of the f-th convolutional kernel, i.e. Among them, s x2 and s y2 The horizontal and vertical convolution strides represent the convolutional strides; Formula (5.3) represents integrating all depthwise convolution results of the f-th convolutional kernel to obtain the depthwise convolution result of the f-th convolutional kernel, i.e. Among them, s z2 The depthwise convolution stride is represented by equation (5.4), which represents the depthwise convolution result that integrates all convolution kernels to obtain the 3D feature representation of this network layer, i.e. The final convolution result is called the 3D CNN interactive feature representation 2, i.e.

[0047] S20405, Obtain 3D CNN Interactive Feature Representation: Connect 3D CNN Interactive Feature Representation 1, i.e. Interactive feature representation 2 with 3D CNN, namely The connection results are then flattened to obtain the final 3D CNN interactive feature representation, i.e. The specific implementation is shown in the following formula:

[0048]

[0049] Wherein, formula (6) represents the connection of the 3D CNN interaction feature representation 1, that is Interactive feature representation 2 with 3D CNN, namely This represents the final 3D CNN interactive feature representation.

[0050] As a preferred embodiment, the specific steps for training the intelligent text reading comprehension model in step S3 are as follows:

[0051] S301. Constructing the loss function

[0052] This invention uses cross-entropy as the loss function;

[0053] S302, Constructing the optimization function

[0054] After testing various optimization functions, the BertAdam optimization function was finally selected as the optimization function for this model. Except for its learning rate being set to 2e-5, all other hyperparameters of BertAdam were set to their default values ​​in PyTorch.

[0055] When the intelligent text reading comprehension model has not yet been trained, it needs to be further trained to optimize its parameters; when the intelligent text reading comprehension model has been trained, it can predict which of the candidate options is the correct option.

[0056] An intelligent text reading comprehension device based on three-dimensional feature representation, the device comprising:

[0057] The intelligent text reading comprehension dataset acquisition unit is used to download publicly available intelligent text reading comprehension datasets from the internet.

[0058] The intelligent text reading comprehension model building unit is used to construct a pre-trained embedding representation module, a feature filtering module, a 3D CNN interactive feature module, and a label prediction module, thereby constructing an intelligent text reading comprehension model.

[0059] The intelligent text reading comprehension model training unit is used to construct the loss function and optimization function to complete the answer selection.

[0060] Preferably, the intelligent text reading comprehension model building unit includes:

[0061] The input module unit is responsible for preprocessing the raw dataset to construct the input data.

[0062] The pre-trained embedding representation module is responsible for using a pre-trained language model to perform embedding encoding operations on the input data, thereby obtaining the embedding representations of articles, questions, and options in the input data.

[0063] The feature filtering module is responsible for performing feature filtering operations on the embedded representations of articles, questions, and options using adaptive average pooling to obtain the corresponding feature pooling representations.

[0064] The 3D CNN interactive feature module unit is responsible for receiving the article pooling feature representation, question pooling feature representation, and option pooling feature representation output by the feature filtering module unit. Then, it uses two interactive feature construction methods to realize the interactive feature construction between the article, question, and option, thereby obtaining two three-dimensional feature representations. Finally, after processing by the 3D CNN structure, the final 3D interactive feature representation is obtained.

[0065] The label prediction module is responsible for determining which candidate option is the correct option based on the final 3D interactive feature representation.

[0066] The intelligent text reading comprehension model training unit also includes:

[0067] The loss function unit is responsible for calculating the error between the prediction result and the actual data using the cross-entropy loss function.

[0068] The optimization function unit is responsible for training and adjusting the parameters during model training to reduce prediction error.

[0069] A storage medium storing multiple instructions, which are loaded by a processor to execute the steps of the above-described intelligent text reading comprehension method based on three-dimensional feature representation.

[0070] An electronic device, the electronic device comprising:

[0071] The aforementioned storage medium; and

[0072] A processor for executing instructions in the storage medium.

[0073] The intelligent text reading comprehension method and apparatus based on three-dimensional feature representation of the present invention have the following advantages:

[0074] (1) This invention can capture and utilize the semantic information contained in the text by pre-training the embedded representation module, making the captured semantic features richer and more accurate;

[0075] (2) The present invention can effectively filter out useless information and reduce sequence length through the feature filtering module, thereby reducing the cost of training the model and improving the efficiency of training.

[0076] (3) The present invention uses a 3D CNN interactive feature module to construct articles, questions and options into a three-dimensional feature form similar to a cube, thereby extracting direct interactive features between the three sequences;

[0077] (4) This invention can capture the direct interaction features between three sequence data through the 3D CNN interaction feature module, and effectively enhance the full interaction between the article, question and option, thereby improving the prediction accuracy of the intelligent text reading comprehension system;

[0078] (5) The method and apparatus proposed in this invention, combined with three-dimensional feature representation, can effectively improve the accuracy of sequence interaction in intelligent text reading comprehension. Attached Figure Description

[0079] The invention will be further described below with reference to the accompanying drawings.

[0080] Figure 1 A flowchart of an intelligent text reading comprehension method based on three-dimensional feature representation.

[0081] Figure 2 Flowchart for building an intelligent text reading comprehension model

[0082] Figure 3 Flowchart for training an intelligent text reading comprehension model

[0083] Figure 4 A flowchart of an intelligent text reading comprehension device based on three-dimensional feature representation.

[0084] Figure 5 A schematic diagram of the structure of the interactive feature module in a 3D CNN.

[0085] Figure 6 This is a schematic diagram of the framework of an intelligent text reading comprehension model based on three-dimensional feature representation. Detailed Implementation

[0086] The intelligent text reading comprehension method and apparatus, storage medium, and electronic device based on three-dimensional feature representation of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0087] Example 1: Intelligent text reading comprehension framework based on three-dimensional feature representation.

[0088] The overall model framework structure of this invention is as follows: Figure 6 As shown. By Figure 6 As can be seen, the main framework of this invention includes a pre-trained embedding representation module, a feature filtering module, a 3D CNN interactive feature module, a 3D CNN interactive feature module, and a label prediction module. The pre-trained embedding representation module performs embedding encoding on the input article sequence, question sequence, and option sequence to obtain their respective embedded encoding representations, which are then passed to the model's feature filtering module. The feature filtering module performs average pooling on the three embedded encoding representations to obtain filtered feature representations, namely, article pooling feature representation, question pooling feature representation, and option pooling feature representation, which are then passed to the 3D CNN interactive feature module. The 3D CNN interactive feature module uses two interactive feature construction methods to construct interactive features between the article, question, and option, resulting in two three-dimensional feature representations. Then, two 3D CNN structures are used to extract the interactive features between the three sequences in the two three-dimensional feature representations, resulting in two 3D interactive feature representations. Finally, these two 3D interactive feature representations are concatenated to obtain the final 3D interactive feature representation, which is then passed to the label prediction module. The label prediction module maps the 3D interactive feature representation to a floating-point value in a specified range, and uses it as the matching degree between the option and the question; then it compares the matching degrees of different options and selects the option with the highest matching degree as the correct answer.

[0089] Example 2: 3D CNN interactive feature module.

[0090] This module receives the article pooling feature representation, question pooling feature representation, and option pooling feature representation output by the feature filtering module as input. First, it processes the article pooling feature representation, question pooling feature representation, and option pooling feature representation using two three-dimensional interaction feature construction methods to obtain two three-dimensional feature representations. Then, it extracts interaction features from these representations using two 3DCNN structures to obtain two 3D CNN interaction feature representations. Finally, it concatenates these two 3D CNN interaction feature representations to obtain the final 3D CNN interaction feature representation.

[0091] Specifically, the implementation process of this module is as follows:

[0092] Step 1: Obtaining the 3D Representation 1: Using the Additive 3D Interactive Feature Construction Method to construct the pooling feature representation of the article. Question Pooling Feature Representation and option pooling feature representation The process is performed to obtain the three-dimensional feature representation 1, i.e. The specific implementation is shown in the following formula:

[0093]

[0094] Formula (2) represents the implementation through additive operations. and The interaction computation among the three, where i and j represent the input tensor. and The element index, I kl and J cl Representing the input tensors respectively and The number of elements in and Indicates the parameters to be trained;

[0095] For example, in PyTorch, the code implementation described above would be as follows:

[0096] query_layer=self.query(context_pool)

[0097] key_layer=self.key(query_pool)

[0098] context_layer=self.context(response_pool)

[0099] key_context=unsqueeze(key_layer,dim=3)+unsqueeze(context_layer,dim=2)

[0100] key_context=key_context.view(key_context.size(0),key_context.size(1),

[0101] key_context.size(2)*key_context.size(3),key_context.size(4))

[0102] query_key_context=unsqueeze(query_layer,dim=3)+unsqueeze(key_context,dim=2)

[0103] query_key_context=query_key_context.view(query_key_context.size(0),

[0104] query_key_context.size(1),query_key_context.size(2)*query_key_context.size(3),query_key_context.size(4))

[0105] query_key_context=self.add_dense(query_key_context)

[0106] Where, context_pool represents the formula query_pool represents the formula in... response_pool represents the formula in... query_key_context represents the key in the formula. `self.query()`, `self.key()`, `self.context()`, and `self.add_dense()` respectively represent the values ​​in the formula. and

[0107] Step 2: Obtaining the 3D Representation 2: Using the Dot 3D Interactive Feature Construction Method to represent the article pooling features. Question Pooling Feature Representation and option pooling feature representation The processing yields a three-dimensional feature representation 2, namely... The specific implementation is shown in the following formula:

[0108]

[0109] i = 1, 2, ..., I kl j = 1, 2, ..., J cl (3)

[0110] Formula (3) represents the implementation through dot product operation. and Interactive computation between the parties, where d represents the input tensor In the index of the element, D represents the input tensor. The number of elements in the input tensor, where i and j represent the number of elements in the input tensor. and The element index, I kl and J cl Representing the input tensors respectively and The number of elements in the tensor; Γ is a third-order tensor with shape D×D×D, where the element at position (d,d,d) has a value of 1 (d=1,2,…,D), and the element at other positions has a value of 0; ×1, ×2 and ×3 represent 1-modular multiplication, 2-modular multiplication and 3-modular multiplication, respectively;

[0111] For example, in PyTorch, the code described above would be implemented as follows: query_layer = self.query(context_pool)

[0112] key_layer=self.key(query_pool)

[0113] context_layer=self.context(response_pool)

[0114] key_context=unsqueeze(key_layer,dim=2)*unsqueeze(context_layer,dim=3)

[0115] key_context=key_context.view(key_context.size(0),key_context.size(1),key_context.size(2)

[0116] *key_context.size(3),key_context.size(4))

[0117] attention_scores=torch.matmul(query_layer,key_context.transpose(-1,-2))

[0118] Where, context_pool represents the formula query_pool represents the formula in... response_pool represents the formula in... attention_scores represents the formula in...

[0119] Step 3: Obtain the 3D CNN interactive feature representation 1: Use a 3D CNN to obtain the three-dimensional feature representation 1, i.e. Interaction feature extraction is performed to obtain the 3D CNN interaction feature representation 1, i.e. The specific implementation is shown in the following formula:

[0120]

[0121]

[0122]

[0123]

[0124] Formula (4.1) indicates that the f-th convolutional kernel represents 1 for the three-dimensional features, i.e. The result of convolving a specific region and mapping it using the ReLU function, where [x1, y1, z1] represents the size of the convolution kernel. This represents the weight matrix of the f-th convolutional kernel, where i, j, and k represent the x-coordinate, y-coordinate, and depth coordinates of the convolutional region, respectively, and m... l m h and m d The three-dimensional feature representation is 1, that is... The length, height, and depth of the region, i: i+x1-1, j: j+y1-1, k: k+z1-1, represent the convolution region. This represents the bias matrix of the f-th convolutional kernel. Let represent the convolution result of the f-th convolutional kernel in the regions i:i+x1-1, j:j+y1-1, k:k+z1-1; Formula (4.2) represents the integration of the horizontal and vertical convolution results of the f-th convolutional kernel in each region to obtain the k-th depth convolution result of the f-th convolutional kernel, where s x1and s y1 Indicates the horizontal convolution stride and the vertical convolution stride. Let represent the depthwise convolution result of the f-th convolution kernel; Equation (4.3) represents integrating all depthwise convolution results of the f-th convolution kernel to obtain the depthwise convolution result of the f-th convolution kernel, where s z1 Indicates the stride of the depthwise convolution. Let f represent the depthwise convolution result of the f-th convolutional kernel; Equation (4.4) represents the integration of the depthwise convolution results of all convolutional kernels to obtain the 3D feature representation of this network layer, i.e. The final convolution result, where, This indicates that the convolutional neural network represents 1 for three-dimensional features, i.e. The final convolution result is called the 3D CNN interactive feature representation.

[0125] For example, in PyTorch, the code implementation described above would be as follows:

[0126] self.conv=Conv3d(in_channels=768, out_channels=16, kernel_size=3, stride=2)

[0127] cqr_conv=self.conv(query_key_context)

[0128] self.classifier=nn.Linear(config.hidden_size*16,1)

[0129] cqr_conv_re1=torch.reshape(cqr_conv,(cqr_conv.view(0),cqr_conv.view(1),cqr_conv.view(2)))

[0130] Wherein, cqr_conv represents 3D feature representation 1, that is, the representation after 3D convolution operation; cqr_conv_re1 represents 3D CNN interactive feature representation 1; 768 represents the input channel dimension, that is, 3D feature representation 1, that is, the last dimension in the representation; 16 represents the number of convolution kernels.

[0131] Step 4: Obtain 3D CNN interactive feature representation 2: Use another 3D CNN to obtain the 3D feature representation 2, i.e. Interaction feature extraction is performed to obtain the 3D CNN interaction feature representation 2, i.e. The specific implementation is shown in the following formula:

[0132]

[0133]

[0134]

[0135]

[0136] Formula (5.1) indicates that the f-th convolutional kernel represents 2 of the three-dimensional features, i.e. The result of convolving a specific region and mapping it using the ReLU function, where [x2, y2, z2] represents the size of the convolution kernel. This represents the weight matrix of the f-th convolutional kernel, where i, j, and k represent the x-coordinate, y-coordinate, and depth coordinates of the convolutional region, respectively, and r... l r h and r d The three-dimensional feature representation is 2, that is The length, height, and depth of the region, i: i+x²-1, j: j+y²-1, and k: k+z²-1 represent the convolution region. This represents the bias matrix of the f-th convolutional kernel. Let f represent the convolution result of the f-th convolutional kernel in the regions i:i+x²-1, j:j+y²-1, and k:k+z²-1; Equation (5.2) represents the integration of the horizontal and vertical convolution results of the f-th convolutional kernel in each region to obtain the k-th depth convolution result of the f-th convolutional kernel, i.e. Among them, s x2 and s y2 The horizontal and vertical convolution strides represent the convolutional strides; Formula (5.3) represents integrating all depthwise convolution results of the f-th convolutional kernel to obtain the depthwise convolution result of the f-th convolutional kernel, i.e. Among them, s z2 The depthwise convolution stride is represented by equation (5.4), which represents the depthwise convolution result that integrates all convolution kernels to obtain the 3D feature representation of this network layer, i.e. The final convolution result is called the 3D CNN interactive feature representation 2, i.e.

[0137] For example, in PyTorch, the code implementation described above would be as follows:

[0138] self.conv=Conv3d(in_channels=768, out_channels=16, kernel_size=3, stride=2)

[0139] cqr_conv=self.conv(attention_scores)

[0140] self.classifier=nn.Linear(config.hidden_size*16,1)

[0141] cqr_conv_re2=torch.reshape(cqr_conv,(cqr_conv.view(0),cqr_conv.view(1),cqr_conv.view(2)))

[0142] Wherein, cqr_conv represents the 3D feature representation 2, that is, the representation after processing by the 3D convolution operation; cqr_conv_re2 represents the 3D CNN interactive feature representation 2; 768 represents the input channel dimension, that is, the last dimension of the 3D feature representation 2; 16 represents the number of convolution kernels.

[0143] Step 5: Obtain the 3D CNN interaction feature representation: Concatenate the 3D CNN interaction feature representation 1, i.e. Interactive feature representation 2 with 3D CNN, namely The connection results are then flattened to obtain the final 3D CNN interactive feature representation, i.e. The specific implementation is shown in the following formula:

[0144]

[0145] Wherein, formula (6) represents the connection of the 3D CNN interaction feature representation 1, that is Interactive feature representation 2 with 3D CNN, namely This represents the final 3D CNN interactive feature representation.

[0146] For example, in PyTorch, the code implementation described above would be as follows:

[0147] cqr_z=torch.cat((cqr_conv_re1,cqr_conv_re2),dim=-1)

[0148] cqr_z = torch.flatten(cqr_z)

[0149] Here, cqr_z is the final 3D CNN interactive feature representation.

[0150] Example 3: Specific steps of the invention.

[0151] As attached Figure 1 As shown, the present invention provides an intelligent text reading comprehension method based on three-dimensional feature representation, which includes the following steps:

[0152] S1. Obtain the intelligent text reading comprehension dataset: Download the publicly available intelligent text reading comprehension dataset from the internet;

[0153] S2. Constructing an intelligent text reading comprehension model: Constructing an intelligent text reading comprehension model using three-dimensional feature representation;

[0154] S3. Train the intelligent text reading comprehension model: Train the intelligent text reading comprehension model constructed in step S2 on the intelligent text reading comprehension training dataset obtained in step S1.

[0155] S1. Obtain the intelligent text reading comprehension dataset.

[0156] Download publicly available intelligent text reading comprehension datasets from the internet or create your own dataset.

[0157] For example, there are many publicly available intelligent text reading comprehension datasets online, such as RACE. The data format in this dataset is as follows:

[0158]

[0159]

[0160] S2. Constructing an intelligent text reading comprehension model

[0161] The process of building an intelligent text reading comprehension model is as follows: Figure 2 As shown, the main operations are to construct an input module, a pre-trained embedding representation module, a feature filtering module, a 3D CNN interactive feature module, and a label prediction module.

[0162] S201, Constructing the Input Module

[0163] For each data point in the dataset, the text sequence is denoted as context; the question sequence is denoted as query; and all candidate options are denoted as response. Based on the correct answer, the label for that data point is determined: if the correct answer is A, it is denoted as 1000; if the correct answer is B, it is denoted as 0100; if the correct answer is C, it is denoted as 0010; and if the correct answer is D, it is denoted as 0001. The three text sequences and the label together constitute one input data point.

[0164] For example, using the data shown in S1 as a sample, we can construct one input data entry. The result is shown below:

[0165] (context:My 8-year-old daughter is making an experiment.She has beenmaking her own colorful smile cards and often takes them with hereverywhere.Last Sunday,I took my kid to go shopping with me.She was hoping tosee John,who is an elderly man and gives out samples.We see him from time totime and he is so happy and friendly.John wasn’t at the store on Sunday,so mydaughter decided that it would be a good idea to distribute her smile cardsto the store’s other employees.So she did.In the produce department,she gavea card to a young man and she hoped it would make him smile.And he smiled ather and thanked her.Then she came across an older gentleman who looked ratherimpatient.And she snuck a card into his cart on top of his groceries,remarking to me later that he looked at her suspiciously as if she wasdumping trash in his cart.But I thought he would be happy later.When we gotback from our shopping trip,she had run out of cards.She was walking by awoman with two babies in her cart.My daughter smiled at her and the youngmother smiled back. My daughter came to me and said excitedly, "Mom, I justrealized something. You don't need cards to make someone smile. All you need to do is make eye contact and smile into their eyes and they will smile back." What a beautiful lesson my daughter reminded me of. writer's daughter theolder gentleman would smile later after he got the smile card we could make others smile only by giving them what they wanted the mother with two babiessmiled because she got a smile card, 0100).

[0166] S202, Constructing pre-trained embedded representation modules

[0167] The pre-trained embedding representation module uses a pre-trained language model to perform embedding encoding operations on the input data constructed in S201, thereby obtaining the embedding representations of the text, questions, and options in the input data, denoted as follows: and The input constructed in S201 contains three text sequences. The article sequence uses a separate encoding module, while the question and option sequences share a common encoding module. Since the pre-trained language model itself contains a multi-layer encoding network, the output of different layers can be selected as its embedding representation based on different input objects. The specific implementation is shown in the following formula:

[0168]

[0169]

[0170]

[0171] In this equation (1.1), the pre-trained language model BERT is used to embed and encode the input article sequence context, and the subscript n represents the number of network layers within BERT. The text represents the embedding representation of the article; Formula (1.2) indicates the embedding encoding of the input question sequence query using the pre-trained language model BERT, where the subscript t indicates the number of network layers within BERT used. This represents the embedded representation of a question; formula (1.3) is basically the same as formula (1.2), except that the encoding object is the option response. This indicates the embedded representation of the option.

[0172] For example, when this invention is implemented on the RACE dataset, n is set to 12 and t is set to 1. In PyTorch, the code implementation described above is as follows:

[0173] context_embed,_=BERT_n(context)

[0174] query_embed, _ = BERT_t(query)

[0175] response_embed,_=BERT_t(response)

[0176] Here, context_embed is the embedded representation of the article, query_embed is the embedded representation of the question, and response_embed is the embedded representation of the option.

[0177] S203, Constructing the Feature Filtering Module

[0178] Embedded representations of articles, questions, and options using adaptive average pooling operations. and Feature filtering is performed to obtain the corresponding pooling feature representations, namely, the article pooling feature representation, the question pooling feature representation, and the option pooling feature representation, which are denoted as follows: and

[0179] For example, in PyTorch, the code implementation described above would be as follows:

[0180] self.context_pool_adapt=AdaptiveAvgPool1d(output_size=64)

[0181] context_pool=torch.transpose(self.context_pool_adapt(torch.transpose(context_embed,dim0=1,dim1=2)),dim0=1,dim1=2)

[0182] self.qr_pool_adapt=AdaptiveAvgPool1d(output_size=15)

[0183] query_pool=torch.transpose(self.qr_pool_adapt(torch.transpose(query_embed,dim0=1,dim1=2)),dim0=1,dim1=2)

[0184] response_pool=torch.transpose(self.qr_pool_adapt(torch.transpose(response_embed,dim0=1,dim1=2)),dim0=1,dim1=2)

[0185] In this code, `AdaptiveAvgPool1d()` represents the adaptive average pooling operation, `output_size` represents the output dimension of the input data after processing by this operation, and in this invention, the output dimension for the article representation is set to 64, and the output dimension for the question and option is set to 15. `torch.transpose` represents the transpose operation, `context_pool` represents the article pooled feature representation obtained after adaptive average pooling of the article's embedding representation, `query_pool` represents the question pooled feature representation obtained after adaptive average pooling of the question's embedding representation, and `response_pool` represents the option pooled feature representation obtained after adaptive average pooling of the option's embedding representation.

[0186] S204. Constructing a 3D CNN interactive feature module

[0187] The module structure is as follows: Figure 5 As shown; its received feature filtering module outputs the article pooling feature representation Question Pooling Feature Representation and option pooling feature representation Then, two interactive feature construction methods are used to construct interactive features among the text, questions, and options, thereby obtaining two three-dimensional feature representations. and Finally, after processing by a 3DCNN structure, the final 3D interactive feature representation is obtained. And then pass it to the label prediction module.

[0188] S205, Constructing a Label Prediction Module

[0189] The final 3D CNN interactive feature representation obtained in step S204 will be used as the input to this module. It will be processed by a fully connected network with a dimension of 4 and an activation function of softmax to obtain the probability of each candidate option being the correct answer. This module will predict the candidate option with the highest probability as the correct answer.

[0190] If the model has not yet been trained, step S3 needs to be executed to train it and optimize the model parameters; when the model has been trained, step S205 predicts which of the options is the correct answer.

[0191] S3. Training an intelligent text reading comprehension model

[0192] The intelligent text reading comprehension model constructed in step S2 is trained on the intelligent text reading comprehension training dataset obtained in step S1. The process is as follows: Figure 3 As shown.

[0193] S301. Constructing the loss function

[0194] This invention uses cross-entropy as the loss function.

[0195] For example, in PyTorch, the code implementation described above would be as follows:

[0196] loss_fct = CrossEntropyLoss()

[0197] loss=loss_fct(logits.view(-1,self.num_labels),labels.view(-1))

[0198] Here, labels are the true labels, and logits are the correct probabilities of the model output.

[0199] S302, Constructing the optimization function

[0200] After testing various optimization functions, the BertAdam optimization function was finally selected as the optimization function for this model. Except for its learning rate being set to 2e-5, all other hyperparameters of BertAdam were set to their default values ​​in PyTorch.

[0201] For example, in PyTorch, the code implementation described above would be as follows:

[0202] optimizer=BertAdam(optimizer_grouped_parameters,lr=2e-5)

[0203] Here, optimizer_grouped_parameters are the parameters to be optimized, which by default are all the parameters in this model.

[0204] If the model has not yet been trained, step S3 needs to be executed to train it and optimize the model parameters; when the model has been trained, step S205 predicts which of the options is the correct answer.

[0205] Example 4:

[0206] Based on Embodiment 1, a storage medium stores multiple instructions, which are loaded by a processor to execute the steps of the intelligent text reading and understanding method based on three-dimensional feature representation in Embodiment 1.

[0207] Example 5:

[0208] An electronic device based on Embodiment 4 includes: a storage medium of Embodiment 4; and a processor for executing instructions in the storage medium.

[0209] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; the present invention has been described in detail with reference to the foregoing embodiments. Those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent text reading comprehension method based on three-dimensional feature representation, characterized in that, The method includes the following steps: S1. Obtain intelligent text reading comprehension datasets: Download publicly available intelligent text reading comprehension datasets from the internet or construct your own dataset; S2. Constructing an intelligent text reading comprehension model: Constructing an intelligent text reading comprehension model based on three-dimensional feature representation; S3. Train the intelligent text reading comprehension model: Train the intelligent text reading comprehension model constructed in step S2 on the intelligent text reading comprehension training dataset obtained in step S1; The specific steps for constructing the intelligent text reading comprehension model in step S2 are as follows: S201, Constructing the Input Module For each data point in the dataset, the text sequence is denoted as context; the question sequence as query; and all candidate options as response. Based on the correct answer, the label for each data point is determined: if the correct answer is A, it is denoted as 1000; if the correct answer is B, it is denoted as 0100; if the correct answer is C, it is denoted as 0010; and if the correct answer is D, it is denoted as 0001. These three text sequences and the label together constitute one input data point. S202, Constructing pre-trained embedded representation modules The pre-trained embedding representation module uses a pre-trained language model to perform embedding encoding operations on the input data constructed in S201, thereby obtaining the embedding representations of the text, questions, and options in the input data, denoted as follows: , and The input constructed in S201 contains three text sequences. The article sequence uses a separate encoding module, while the question and option sequences share a common encoding module. Since the pre-trained language model itself contains a multi-layer encoding network, the output of different layers can be selected as its embedding representation based on different input objects. See the following formula for specific implementation: （1.1） （1.2） （1.3） Wherein, formula (1.1) represents the use of a pre-trained language model BERT For the input article sequence Embedded encoding, subscript n Indicates the use BERT Internal network layers The embedding representation of the article is given; Equation (1.2) indicates the use of a pre-trained language model. BERT For the input sequence of questions Embedded encoding, subscript t Indicates the use BERT Internal network layers The embedded representation of the question is shown; Formula (1.3) is basically the same as Formula (1.2), except that the encoding object is the option. , This indicates an embedded representation of the options; S203, Constructing the Feature Filtering Module Embedded representations of articles, questions, and options using adaptive average pooling operations. , and Feature filtering is performed to obtain the corresponding pooling feature representations, namely, the article pooling feature representation, the question pooling feature representation, and the option pooling feature representation, which are denoted as follows: , and ; S204. Constructing a 3D CNN interactive feature module Receive the article pooling feature representation output by the feature filtering module Question pooling feature representation and option pooling feature representation Then, two interactive feature construction methods are used to construct interactive features among the article, questions, and options, thus obtaining two three-dimensional feature representations. and Finally, after processing by a 3D CNN structure, the final 3D interactive feature representation is obtained. ; S205, Constructing a Label Prediction Module The final 3D interactive feature representation obtained in step S204 The input will be used as the input to this module, and it will be processed by a fully connected network with a dimension of 4 and an activation function of softmax to obtain the probability of each candidate option as the correct answer. This module will predict the candidate option with the highest probability as the correct answer. If the model has not yet been trained, step S3 needs to be executed to train it and optimize the model parameters; when the model has been trained, step S205 predicts which of the options is the correct answer.

2. The intelligent text reading comprehension method based on three-dimensional feature representation according to claim 1, characterized in that, The 3D CNN interactive feature module construction process in step S204 is as follows: S20401. Obtaining the 3D Representation 1: Using the Additive 3D Interactive Feature Construction Method to Represent the Article Pooling Features Question pooling feature representation and option pooling feature representation The process is performed to obtain the three-dimensional feature representation 1, i.e. The specific implementation is shown in the following formula: ， i=1,2,…,i kl j=1,2,…,J cl （2） Formula (2) represents the implementation through additive operations. , and Interactive computation among the three i and j Indicates the input tensor and element index, I kl and J cl Representing the input tensors respectively and The number of elements in , , and Indicates the parameters to be trained; S20402. Obtaining the 3D Representation 2: Using the Dot 3D Interactive Feature Construction Method to Represent the Article Pooling Features Question pooling feature representation and option pooling feature representation The processing yields a three-dimensional feature representation 2, namely... The specific implementation is shown in the following formula: ， i=1,2,…,i kl j=1,2,…,J cl （3） Formula (3) represents the implementation through the Dot product operation. , and Interactive computation among the three d Indicates the input tensor element index in D Indicates the input tensor The number of elements in i and j Indicates the input tensor and element index, I kl and J cl Representing the input tensors respectively and The number of elements in; It is a shape of D D D The third-order tensor, where the position is ( d,d,d The element value of ) is 1. d The value range is 1, 2, ..., D The values ​​of all other elements are 0; , and These represent modular multiplication (1-module), modular multiplication (2-module), and modular multiplication (3-module), respectively. S20403, Obtaining 3D CNN Interactive Feature Representation 1: Using a 3D CNN (3D Convolutional Neural Network) to obtain the 3D feature representation 1, i.e. Interaction feature extraction is performed to obtain the 3D CNN interaction feature representation 1, i.e. The specific implementation is shown in the following formula: （4.1） （4.2） （4.3） （4.4） Formula (4.1) represents the first... f Each convolutional kernel represents 1 for a three-dimensional feature, that is... After convolution of a specific region ReLU The result of the function mapping, where, [ x 1 , y 1 , z 1 The ] indicates the size of the convolution kernel. Indicates the first f The weight matrix of each convolutional kernel. i , j and k Represents the x-coordinate, y-coordinate, and depth coordinates of the convolution region. , and The three-dimensional feature representation is 1, that is... The length, height and depth i:i + x 1 -1, j:j + y 1 -1, k:k + z 1 -1 indicates the convolution region. Indicates the first f The bias matrix of each convolution kernel. Indicates the first f Each convolutional kernel in i:i + x 1 -1, j:j + y 1 -1, k:k + z 1 The convolution result in the -1 region; Formula (4.2) represents the integration of the first... f The convolutional kernel performs horizontal and vertical convolutions in each region to obtain the first convolutional kernel. f The first convolutional kernel k The result of depthwise convolution, where, and Indicates the horizontal convolution stride and the vertical convolution stride. Indicates the first f The first convolutional kernel k The result of depthwise convolution; Equation (4.3) represents the integration of the first... f To obtain the depthwise convolution results of the nth convolution kernel, we can use the results of all depthwise convolutions of the nth convolution kernel. f The depthwise convolution result of each convolution kernel, where, Indicates the stride of the depthwise convolution. Indicates the first f The depthwise convolution result of each convolutional kernel; Formula (4.4) represents the integration of the depthwise convolution results of all convolutional kernels to obtain the three-dimensional feature representation of this layer, i.e. The final convolution result, where, This indicates that the convolutional neural network represents 1 for three-dimensional features, i.e. The final convolution result is called the 3D CNN interactive feature representation. S20404, Obtaining 3D CNN Interactive Feature Representation 2: Using another 3D CNN for 3D feature representation 2, i.e. Interaction feature extraction is performed to obtain the 3D CNN interaction feature representation 2, i.e. The specific implementation is shown in the following formula: （5.1） （5.2） （5.3） （5.4） Formula (5.1) represents the first... f Each convolutional kernel represents 2 three-dimensional features, i.e. After convolution of a specific region ReLU The result of the function mapping, where, [ x 1 , y 1 , z 1 The ] indicates the size of the convolution kernel. Indicates the first f The weight matrix of each convolutional kernel. i , j and k Represents the x-coordinate, y-coordinate, and depth coordinates of the convolution region. , and The three-dimensional feature representation is 2, that is The length, height and depth i:i + x 2 -1, j:j + y 2 -1, k:k + z 2 -1 indicates the convolution region. Indicates the first f The bias matrix of each convolution kernel. Indicates the first f Each convolutional kernel in i:i + x 2 -1, j:j + y 2 -1, k:k + z 2 The convolution result in the -1 region; Formula (5.2) represents the convolution result of the -1 region. f The convolutional kernel performs horizontal and vertical convolutions in each region to obtain the first convolutional kernel. f The first convolutional kernel k The result of depthwise convolution, i.e. ,in, and The horizontal convolution stride and the vertical convolution stride are represented; Formula (5.3) represents the integration of the first... f To obtain the depthwise convolution results of the nth convolution kernel, we can use the results of all depthwise convolutions of the nth convolution kernel. f The depthwise convolution result of each convolution kernel, i.e. ,in, The depthwise convolution stride is represented by equation (5.4), which represents the depthwise convolution result that integrates all convolution kernels to obtain the 3D feature representation of this network layer, i.e. The final convolution result is called the 3D CNN interactive feature representation 2, i.e. ; S20405, Obtain 3D CNN Interactive Feature Representation: Connect 3D CNN Interactive Feature Representation 1, i.e. Interactive feature representation 2 with 3D CNN, namely The connection results are then flattened to obtain the final 3D CNN interactive feature representation. The specific implementation is shown in the following formula: （6） Wherein, formula (6) represents the connection of the 3D CNN interactive feature representation 1, that is Interactive feature representation 2 with 3D CNN, namely , This represents the final 3D CNN interactive feature representation.

3. The intelligent text reading and understanding method based on three-dimensional feature representation according to claim 1, characterized in that, The specific steps for training the intelligent text reading comprehension model in step S3 are as follows: S301. Constructing the loss function Cross-entropy is used as the loss function; S302, Constructing the optimization function After testing various optimization functions, the model was ultimately selected to use... Bert The Adam optimization function is used as the optimization function for this model, except that its learning rate is set to 0.00002. Bert All other hyperparameters of Adam were set to their default values ​​in PyTorch. If the model has not yet been trained, step S3 needs to be performed to train it and optimize the model parameters; once the model has been trained, step S2 predicts which option is the correct answer.

4. An intelligent text reading comprehension device based on three-dimensional feature representation, implementing the intelligent text reading comprehension method based on three-dimensional feature representation as described in any one of claims 1 to 3, characterized in that, The device includes: The intelligent text reading comprehension dataset acquisition unit is used to download publicly available intelligent text reading comprehension datasets from the internet or to construct its own datasets. The intelligent text reading comprehension model building unit is used to build a pre-trained embedding representation module, a feature filtering module, a 3DCNN interactive feature module, and a label prediction module, thereby building an intelligent text reading comprehension model; The intelligent text reading comprehension model training unit is used to construct the loss function and optimization function to complete the answer selection.

5. The intelligent text reading and comprehension device based on three-dimensional feature representation according to claim 4, characterized in that, The intelligent text reading comprehension model construction unit includes: The input module is responsible for preprocessing the raw dataset to construct the input data; The pre-trained embedding representation module is responsible for using a pre-trained language model to perform embedding encoding operations on the input data, thereby obtaining the embedding representations of articles, questions, and options in the input data; The feature filtering module is responsible for performing feature filtering operations on the embedded representations of articles, questions, and options using adaptive average pooling to obtain the corresponding pooled feature representations. The 3D CNN interactive feature module is responsible for receiving the article pooling feature representation, question pooling feature representation, and option pooling feature representation output by the feature filtering module. Then, it uses two interactive feature construction methods to construct interactive features between the article, question, and option, thereby obtaining two three-dimensional feature representations. Finally, after processing by the 3D CNN structure, the final 3D interactive feature representation is obtained. The label prediction module is responsible for determining which candidate option is the correct option based on the final 3D interactive feature representation. The intelligent text reading comprehension model training unit also includes: The loss function is responsible for calculating the error between the prediction result and the actual data using the cross-entropy loss function. The optimization function is responsible for training and adjusting the parameters during model training to reduce prediction error.

6. A storage medium storing a plurality of instructions, characterized in that, The instructions are loaded by the processor and execute the intelligent text reading and understanding method based on three-dimensional feature representation as described in any one of claims 1 to 3.

7. An electronic device, characterized in that, The electronic device includes: The storage medium as claimed in claim 6; and A processor for executing instructions in the storage medium.

Images