Intelligent question answering method based on natural language processing and deep learning algorithm

Through an intelligent question-and-answer method based on natural language processing and deep learning algorithms, the question classification model is constructed using the RoBERTa-DPCNN model and the BERT language model, and the inaccurate information retrieval caused by the semantic complexity of "non-grain" data of cultivated land is solved, and the problem of inaccurate information retrieval caused by the semantic complexity of cultivated land is realized. Efficient and accurate information retrieval effect.

CN118981518AActive Publication Date: 2024-11-19SICHUAN PROVINCIAL INST OF LAND SCI & TECH (SICHUAN PROVINCIAL SATELLITE APPL TECH CENT) +1

Patent Information

Application Number
CN202410992678.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-19
Estimated Expiration
2044-07-23

AI Technical Summary

Technical Problem

The data related to "non-grainization" of cultivated land is large in volume and complex in semantics. It is difficult to achieve accurate information retrieval by traditional keyword matching and fixed syntactic structure, resulting in inaccurate query results.

Method used

Using intelligent question-answer methods based on natural language processing and deep learning algorithms, the question classification model is constructed through the RoBERTa-DPCNN model, the sentence expression forms of the "non-grain" monitoring data related to cultivated land are analyzed, and the question-sample library is constructed using the BERT language model. Combining deep local feature extraction and full connection layer, we realize question type discrimination.

Benefits of technology

It realizes accurate and efficient query of information retrieval in the field of "non-grainization" arable land, overcomes the semantic complexity and diversity problems in traditional methods, and improves the accuracy and efficiency of query results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118981518A_ABST
    Figure CN118981518A_ABST
Patent Text Reader

Abstract

The invention provides an intelligent question and answer method based on natural language processing and a deep learning algorithm, and the method comprises the steps: obtaining the monitoring result data of a cultivated land in a certain region, analyzing the internal relation and numerous quantitative indexes between the non-grain related resources of the cultivated land, and determining the non-grain related semantic query problem of the cultivated land; and a cultivated land non-grain question sample library and a question classification model are constructed. The RoBERTa-DPCNN question classification model constructed by the method is based on a statement expression form of knowledge query of cultivated land non-grain related monitoring data, and information retrieval can be accurately and efficiently performed in the non-grain field.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computer natural language processing, and specifically relates to an intelligent question-answering method based on natural language processing and deep learning algorithms. Background Art

[0002] The non-grainization of cultivated land has always been a major issue that cannot be avoided in food security. In order to prevent the non-grainization of cultivated land, firmly protect the lifeline of national food security, and build a "three-in-one" protection and supervision system for cultivated land quantity, quality, and ecology, the national and local natural resources departments have conducted unified surveys and monitoring, and generated and accumulated a large amount of survey and monitoring data.

[0003] However, the data involved in the "non-grainization" of cultivated land is huge and complex in structure, and the corresponding search statements have high semantic complexity and diversity. If information retrieval only relies on traditional keyword matching and fixed syntactic structures, inaccurate query results may result.

[0004] Therefore, it is necessary to conduct in-depth analysis and design of the question types in the intelligent question-answering of "non-grainization" of cultivated land, so as to quickly and accurately obtain the semantic information in natural language questions, thereby realizing intelligent and efficient question-answering queries. Summary of the invention

[0005] The purpose of the present invention is to propose an intelligent question-answering method based on natural language processing and deep learning algorithms to solve the problems of existing question-answering queries raised in the above-mentioned background technology. This method is based on the sentence expression form of knowledge query of relevant monitoring data on the "non-grainization" of cultivated land, and can accurately and efficiently perform information retrieval in the field of "non-grainization".

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] An intelligent question answering method based on natural language processing and deep learning algorithm includes the following steps:

[0008] Step 1: Obtain the monitoring data of cultivated land in a certain area, analyze the internal connection between the resources related to the “non-grainization” of cultivated land and many quantitative indicators, and determine the semantic query problems related to the “non-grainization” of cultivated land;

[0009] Step 2: Construct a sample database of questions about “non-grainization” of cultivated land;

[0010] Step 3: Build a question classification model.

[0011] Furthermore, the semantic query questions include but are not limited to the following eight categories, namely geographic location query, inclusion query, cultivated land type change query, cultivated land outflow query within the scope of permanent basic farmland, cultivated land outflow query within the scope of cultivated land protection target, new construction and suspected illegality involved in cultivated land outflow, ecological protection red line occupation query and related indicator comparison query.

[0012] Furthermore, the step 2 uses the BERT language model to construct a question sample library.

[0013] Furthermore, the question classification model is a RoBERTa-DPCNN model. The core architecture of the RoBERTa-DPCNN model includes a text input layer, a word vector representation layer, a deep local feature extraction layer and an output layer. The deep local feature extraction layer uses a deep pyramid convolutional neural network DPCNN to deeply extract feature information from the deep local feature representation layer. The RoBERTa-DPCNN model is based on the pre-trained language model RoBERTa. The features extracted by the RoBERTa model are sent to the DPCNN model, and then the obtained word vector is processed with the help of a fully connected layer. The Softmax function is used to realize the type discrimination of the question and output the classification result.

[0014] Furthermore, the training and parameter tuning process of the RoBERTa-DPCNN model includes the following steps: constructing a question training sample library according to the classification of the question "non-grainization" of cultivated land; constructing training data, and dividing the training set and the test set according to a 7:3 ratio on the basis of maintaining the original appearance of the question sample library; loading the RoBERTa-large pre-trained model for pre-training on large-scale text data, and fine-tuning on the question text data; conducting in-depth training and parameter tuning of the RoBERTa-DPCNN model; and evaluating and analyzing the performance of the RoBERTa-DPCNN model in the question classification task.

[0015] The present invention has the following beneficial effects:

[0016] The resources related to the "non-grainization" of cultivated land have the characteristics of large data volume and complex semantic relationships. The corresponding search sentences have high semantic complexity and diversity. Relying only on traditional keyword matching and fixed syntactic structure for information retrieval will lead to inaccurate query results. The present invention constructs a RoBERTa-DPCNN question classification model, which is based on the sentence expression form of knowledge query of the monitoring data related to the "non-grainization" of cultivated land, and can accurately and efficiently perform information retrieval in the field of "non-grainization". BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0018] Figure 1 It is the RoBERTa-DPCNN model structure;

[0019] Figure 2 It is the DPCNN model structure;

[0020] Figure 3 This is the convergence diagram of the RoBERTa-DPCNN question classification model. DETAILED DESCRIPTION

[0021] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] See also Figure 1-3 As shown, an intelligent question answering method based on natural language processing and deep learning algorithm includes the following steps:

[0023] Step one, obtaining the monitoring results data of cultivated land in a certain area. The present invention is based on the monitoring results data of cultivated land in Sichuan Province, analyzes the internal connection between the resources related to the "non-grainization" of cultivated land and many quantitative indicators, and designs semantic query problems related to the "non-grainization" of cultivated land.

[0024] When making relevant queries on the monitoring results data of "non-grainization" of cultivated land, natural language questions are usually interrogative sentences or declarative sentences, such as "What is the area of ​​non-grainization of cultivated land outflow type in Chengdu in 2022?" is queried in the form of an interrogative sentence; "Query the area of ​​non-grainization of cultivated land outflow type in Chengdu in 2022" is queried in the form of a declarative sentence. The present invention is based on the sentence expression form of knowledge query of the monitoring data related to "non-grainization" of cultivated land, analyzes the intrinsic connection between the resources related to "non-grainization" of cultivated land and many quantitative indicators, and designs 8 types of semantic query questions related to "non-grainization" of cultivated land, as shown in Table 1.

[0025] Table 1 Statistics of query statements related to “non-grainization” of cultivated land

[0026]

[0027]

[0028] Step 2: Construct a sample library of questions on “non-grainization” of cultivated land.

[0029] When constructing a sample library of questions about the “non-grainization” of cultivated land, it is necessary to consider the coverage, diversity, quality, annotation accuracy of the question samples and the requirements of meeting people's language expression habits. Based on the knowledge problem system of “non-grainization” of cultivated land, the present invention constructs an experimental sample database using synonym replacement technology. Considering that replacement based on the language model can generate new samples with more diversity and semantic coherence, the present invention uses the BERT (Bidirectional Encoder Representations from Transformers) language model to construct a sample library of questions. Taking “How many acres of non-grain areas have flowed out of the cultivated land protection target area in Chengdu in 2022?” as an example, through the synonym replacement of the language model, new question samples such as “In 2022, what is the area of ​​land converted to non-grain use in the cultivated land protection target area in Chengdu?” and “How large is the area of ​​non-grain land in the cultivated land protection target area in Chengdu in 2022?” can be obtained. The constructed sample library of questions contains 4592 samples. The detailed information of the sample library is shown in Table 2.

[0030] Table 2 Data volume of question sample library

[0031]

[0032] Step 3: Build a question classification model.

[0033] In order to enable the computer to accurately identify the category of natural language questions related to the "non-grainization" of cultivated land raised by users, and generate corresponding database query instructions to implement data retrieval, building a question classification model is an important step to achieve accurate conversion of natural language questions into computer language. In order to overcome the long-distance dependency problem of text in the commonly used convolutional neural network CNN (Convolutional Neural Network), the present invention constructs a RoBERTa-DPCNN model, which is based on the pre-trained language model RoBERTa (Robustly optimized BERT approach) and plays a core role in feature extraction. In order to obtain a high-level semantic representation of the text and accurately capture semantic information, the features extracted by the RoBERTa model are sent to the DPCNN model to further extract the deep local features of the sentence, and then the word vector obtained by processing with the help of the fully connected layer is used. The Softmax function is used to realize the type discrimination of the question and output the classification result. The core architecture of the model includes a text input layer, a word vector representation layer, a deep local feature extraction layer and an output layer. The structural block diagram of the model is shown as follows: Figure 1 shown.

[0034] In order to achieve the vectorized representation of the initial text input, the present invention uses the pre-trained language model RoBERTa (Robustly optimized BERT approach) for processing. The RoBERTa model is improved on the basis of the BERT model. The RoBERTa model uses a longer pre-training time, a larger batch size and more training data, and deletes the NSP (Next Sentence Prediction) task in the BERT model to improve the generalization ability of the model. In addition, the RoBERTa model also adopts a dynamic masking strategy, randomly selecting a part of the tokens for masking in each training iteration, instead of fixedly replacing 15% of the tokens with [MASK]. This model can convert text input into an efficient vector representation, and in text processing tasks in specific fields, subsequent models can show excellent performance.

[0035] In the deep local feature extraction layer, the present invention uses a deep pyramid convolutional neural network (DPCNN) to perform in-depth feature information extraction on the deep local feature representation layer. DPCNN can be regarded as an optimization and extension of CNN. In the field of text classification, compared with traditional convolutional neural networks, DPCNN significantly enhances the ability to capture dependencies between long-distance text sequences, thereby effectively improving the processing efficiency and performance of text classification tasks. Specifically, the network architecture of DPCNN is as follows: Figure 2 As shown in the figure. Its regional embedding layer designs convolution filters of various sizes, aiming to perform word vector encoding on the input text data through convolution operations. In the model structure of DPCNN, there are two convolution blocks, each of which consists of two convolution layers, and uses pre-activation technology to optimize performance. After the convolution block is the pooling layer, which performs a 1 / 2 pooling operation, that is, downsampling. With the cyclic convolution and sampling of the model, a pyramid structure model is formed.

[0036] Step 4: Train the question classification model and optimize its parameters.

[0037] The training and parameter tuning process of the RoBERTa-DPCNN question classification model can be summarized into five main steps, including: 1. Construct a question training sample library based on the classification of questions about "non-grainization" of cultivated land; 2. Construct training data, and divide the training set and test set into 7:3 ratios while maintaining the original appearance of the question sample library; 3. Load the RoBERTa-large pre-trained model for pre-training on large-scale text data, and fine-tune it on its own question text data; 4. Conduct in-depth training and parameter tuning of the RoBERTa-DPCNN model; 5. Performance evaluation and analysis of the question classification task of the RoBERTa-DPCNN model. Among them, fine-tuning the pre-trained model in step 3 can reduce training time and resource costs, and step 4 will directly affect the question classification accuracy of the RoBERTa-DPCNN model. The main parameters that need to be adjusted and set for the RoBERTa-DPCNN model are shown in Table 3. Model convergence analysis is shown in Table 3. Figure 3 As shown in the figure, the horizontal axis represents the change of learning rate (Learning_rate), and the vertical axis shows the corresponding performance of recall rate (Recall), accuracy rate (Precision) and F1 value under different learning rate settings. The graphical analysis results show that when the learning rate is adjusted to above 0.23, the Recall, Precision and F1 values ​​all show a stable trend, and no further significant fluctuations are observed. Based on this, it is determined that the optimal learning rate of the model should be set to 0.22.

[0038] Table 3 RoBERTa-DPCNN model parameter settings

[0039]

[0040] In order to verify the effectiveness of the question classification model on the test set, after the RoBERTa-DPCNN question classification model is constructed, it is necessary to evaluate the question classification performance of the model. The present invention selects to use three indicators of accuracy, recall rate and F1 value to evaluate the performance of the RoBERTa-DPCNN question classification model. The calculation formulas of accuracy, recall rate and F1 value are shown in formulas (1) to (3).

[0041]

[0042] In the formula, TP means that the question classification model predicts that the sample is actually a positive sample, FP means that the question classification model predicts that the sample is actually a positive sample, and FN means that the question classification model predicts that the sample is actually a negative sample. The higher the values ​​of the three indicators, the better the performance of the question classification model. The test set extracted from the question sample library is used to verify the classification effect of the question classification model.

[0043] In order to accurately evaluate the effectiveness of the question classification model, this paper uses a comparative analysis method for in-depth discussion. At the experimental operation level, relying on a uniformly constructed question sample database, the same data set is selected as the basis for training and testing, and the RoBERTa-DPCNN model, BERT-DPCNN model, RoBERTa model, and TextCNN model are systematically trained and evaluated. In order to comprehensively evaluate the performance of the model, three key indicators, accuracy, recall rate, and F1 value, are selected, and a comparative analysis is conducted accordingly. The specific comparative experimental results are shown in Table 4.

[0044] Table 4. Comparative experimental results of question classification models

[0045]

[0046] By analyzing the model indicator values ​​in Table 4, it can be seen that the RoBERTa-DPCNN model has the best effect on question classification, among which the RoBERTa-DPCNN model performs better than the BERT-DPCNN model and the RoBERTa model performs better than the BERT model. The main reason is that the RoBERTa model has greatly improved its performance relative to the BERT model through improvements in five aspects: mask strategy, next sentence prediction task, expansion of pre-training data, pre-training time, and batch size. The classification effect of the RoBERTa-DPCNN model is better than that of the RoBERTa model and the BERT-DPCNN model is better than that of the BERT model. The main reason is that the DPCNN model has advantages in feature extraction and a more complex model combination structure has better generalization ability.

[0047] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well.

Claims

1. An intelligent question-answering method based on natural language processing and deep learning algorithm, characterized in that: The following steps are involved: Step 1: Obtain the monitoring data of cultivated land in a certain area, analyze the internal connection between the resources related to "non-grainization" of cultivated land and many quantitative indicators, and determine the semantic query problems related to "non-grainization" of cultivated land; Step 2: Construct a sample database of questions about "non-grainization" of cultivated land; Step 3: Build a question classification model.

2. The intelligent question-answering method based on natural language processing and deep learning algorithm according to claim 1, characterized in that: The semantic query questions include but are not limited to the following eight categories, namely, geographic location query, inclusion query, cultivated land type change query, cultivated land outflow query within the scope of permanent basic farmland, cultivated land outflow query within the scope of cultivated land protection target, new construction and suspected illegality involved in cultivated land outflow, ecological protection red line occupation query and related indicator comparison query.

3. The intelligent question-answering method based on natural language processing and deep learning algorithm according to claim 2, characterized in that: The step 2 uses the BERT language model to build a question sample library.

4. The intelligent question-answering method based on natural language processing and deep learning algorithm according to claim 3, characterized in that: The question classification model is a RoBERTa-DPCNN model. The core architecture of the RoBERTa-DPCNN model includes a text input layer, a word vector representation layer, a deep local feature extraction layer and an output layer. The deep local feature extraction layer uses a deep pyramid convolutional neural network DPCNN to deeply extract feature information from the deep local feature representation layer. The RoBERTa-DPCNN model is based on the pre-trained language model RoBERTa. The features extracted by the RoBERTa model are sent to the DPCNN model, and then the word vectors obtained are processed with the help of a fully connected layer. The Softmax function is used to realize the type discrimination of the question and output the classification result.

5. The intelligent question-answering method based on natural language processing and deep learning algorithm according to claim 4, characterized in that: The training and parameter tuning process of the RoBERTa-DPCNN model includes the following steps: constructing a question training sample library according to the classification of the question "non-grainization of cultivated land"; constructing training data, and dividing the training set and the test set according to a 7:3 ratio on the basis of keeping the original appearance of the question sample library; loading the RoBERTa-large pre-trained model for pre-training on large-scale text data, and fine-tuning on the question text data; conducting in-depth training and parameter tuning of the RoBERTa-DPCNN model; and evaluating and analyzing the performance of the RoBERTa-DPCNN model in the question classification task.

Citation Information

Patent Citations

  • Commodity question and answer method based on RoBERTa-WWM

    CN115994209A

  • Multi-intention recognition method and device

    CN116244412A

  • Question and answer method and system based on RoBERTa-DPCNN model

    CN117149966A

  • Text classification model training method, text classification method, apparatus, device, storage medium and computer program product

    US20230025317A1

Cited By

  • Science and technology public text intelligent classification and service method and device based on deep learning

    CN120653775A

  • Deep learning-based technology public text intelligent classification and service method and device

    CN120653775B