Intelligent question answering system and method based on natural language processing

Abstract

The invention relates to the technical field of natural language processing, and discloses an intelligent question-answering system and method based on natural language processing, and the system comprises an information processing input module, a user intention analysis module, a question-answering statement selection module and a question-answering statement feedback module. The information processing input module comprises a voice input unit, a text input unit and an image input unit. According to the system, a first input statement is obtained by carrying out preliminary information processing input on a voice file or an image file, a text file is directly used as the first input statement, then the first input statement is processed through a text input unit, a second input statement is generated, all key information of the second input statement is extracted, and then the first input statement and the second input statement are stored. The statement keyword is output, the user statement key information is obtained through analysis, the question and answer statement is output based on the input question and answer statement database and the user statement key information, and therefore it is guaranteed that the keyword of the user input content is fully extracted, and the answer accuracy is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and specifically to an intelligent question-answering system and method based on natural language processing. Background Art

[0002] With the rapid development of the Internet and the widespread popularity of personal computers, more and more messages and data are published in the form of electronic documents through the hypertext transfer protocol. As a result, the speed and ability of data retrieval have encountered huge challenges. How to accurately and timely obtain the information required by users in the vast world of information has become a major problem in the development of the Internet.

[0003] Search engine technology is a relatively mature information retrieval technology. However, with the crazy growth of Internet data, the shortcomings of search engines have gradually emerged. Traditional search engines such as Baidu, Google, and Bing usually only take keywords as input. For ordinary users, it is often difficult for them to condense a small number of keywords to accurately express their query intentions. In addition, the results returned by the search engine are not a concise and accurate answer, but a list of web page fragments. These web page fragments usually contain a lot of noise data, and users still need to read these web page fragments or even the corresponding original web pages to find the answers they need.

[0004] In order to improve the user experience of information retrieval, people began to study question-answering systems that use natural language as input and output. Users can directly express their query needs in natural language in the form of text or voice. After the question-answering system understands the user's query intention, it directly returns the accurate answer expressed in natural language to the user through a series of retrieval, analysis and processing. Therefore, for users, the question-answering system is a more convenient, friendly and accurate service.

[0005] For enterprises with manual customer service, the Q&A system can save a lot of manpower for them, and the Q&A system is more stable and efficient. For example, China Mobile's traditional customer service methods include 10086 manual service, business hall manual service window and other channels. These service methods have communication fees, training fees, labor resources and other costs, and are subject to time (unable to provide 24-hour service) and venue (centralized customer service office space) and other conditions. With the growth of the number of corporate customers, the huge consulting needs often overwhelm the customer service team. Therefore, under the wave of enterprise modernization, informatization and intelligent development, intelligent Q&A systems have emerged. Existing Q&A systems extract keywords from uploaded content and retrieve corresponding answers from the database based on keywords. However, when users upload a lot of content, or when users upload voice or image files, they cannot obtain the most appropriate keywords from more content, resulting in inaccurate answers. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention provides an intelligent question-answering system and method based on natural language processing, which has the advantages of extracting keywords and accurately answering, and solves the above-mentioned technical problems.

[0007] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: an intelligent question-answering system based on natural language processing, comprising an information processing input module, a user intention analysis module, a question-answering sentence selection module and a question-answering sentence feedback module; The information processing and input module includes a voice input unit, a text input unit and an image input unit. The information processing and input module identifies all information uploaded by the user, and classifies the information according to the format of the information uploaded by the user to obtain classification results: voice files, text files and image files, and assigns the information to one of the voice input unit and the image input unit according to the classification results for information processing and input to obtain a first input sentence, and processes the first input sentence through the text input unit to generate a second input sentence; The user intention analysis module includes a user information extraction unit and a user intention output unit. The user information extraction unit executes a semantic extraction instruction after reading the second input sentence, extracts all key information of the second input sentence, and outputs the sentence keywords. , and executes the information analysis instruction, the information analysis instruction calls the user intention output unit, and based on the sentence keywords , analyze and obtain key information of user sentences ; The question and answer sentence selection module outputs the question and answer sentence based on the entered question and answer sentence database and the key information of the user sentence, displays it to the user, and waits for the user's feedback coefficient The question-answering sentence feedback module is based on the user feedback coefficient Output an adjustment instruction, wherein the adjustment instruction adjusts the user information extraction unit before the next round of question and answer begins.

[0008] As a preferred technical solution of the present invention, the specific steps of the voice input unit generating the input sentence are as follows: Step A1: De-noise the voice file and divide the de-noised voice file into frames. The specific expression is as follows: ; in, Indicates the first Voice samples, The first samples, Indicates Window length; Step A2: After splitting the frame Voice samples Classification is performed and a first entry statement is generated.

[0009] As a preferred technical solution of the present invention, the specific steps of the image input unit processing the image file to generate the first input sentence are as follows: Step B1: scaling and graying the image file, converting the gray image into a black and white image, and extracting the text area by threshold segmentation; Step B2: Analyze the connectivity of image pixels and remove image noise; Step B3: Recognize the text after noise removal through an optical character recognition model, and arrange the recognition results in order to obtain a first input sentence.

[0010] As a preferred technical solution of the present invention, the specific expression of converting the grayscale image into a black and white image in step B1 and extracting the text area by threshold segmentation is as follows: ; in, Represents pixel The qualified value of Represents pixel For the text area, Represents pixel The gray value of Represents the pixel threshold.

[0011] As a preferred technical solution of the present invention, the specific steps of step B2 for marking the text blocks by analyzing the connectivity of the image pixels are as follows: Step B2.1: Select all the areas in the image that meet the requirements of step B1; Step B2.2: Comprehensively score the selected parts ; The overall score The specific calculation expression is as follows: ; in, and represents the weight coefficients that sum to 1, Indicates the length of the selected area. Indicates the width of the selection area. Indicates the area of ​​the selected area. Indicates the maximum area of ​​the selected area, which is the preset value for recognition; Step B2.3: Comprehensive scoring The part that meets the judgment condition is outputted and arranged from top to bottom according to the position of the frame selection center to obtain the first input statement.

[0012] As a preferred technical solution of the present invention, the text input unit processes the first input sentence to generate the second input sentence in the following specific steps: Step C1: read the connection word library in the question and answer sentence database, distinguish the nature of the words in the first input sentence, and remove prepositions and adjectives; Step C2: Split the first input sentence after elimination according to punctuation marks to obtain multiple segments, and use all the segments as the second input sentence.

[0013] As a preferred technical solution of the present invention, the user information extraction unit executes a semantic extraction instruction based on the second input sentence, extracts all key information of the second input sentence, and outputs the sentence keywords. The specific expression is as follows: ; in, Represents three different sentence keywords for output, and the acquisition steps are as follows: Step D1: extract all the words at the beginning of the second input sentence to form a first keyword library, extract all the words at the end of the second input sentence to form a second keyword library, and extract all the repeated words in the second input sentence to form a third keyword library; Step D2: Read the first keyword from the third keyword library Words , and read the Words Number of times in the first keyword library The number of times the second keyword library appears ; Step D3: Calculate the Words The key factor Until the traversal of the third keyword library is completed, the three with the largest key coefficients are selected as three different sentence keywords .

[0014] As a preferred technical solution of the present invention, the step D3 calculates Words The key factor The expression is as follows: ; in, Indicates Words The number of times in the first keyword library, Indicates Words The number of times it appears in the second keyword library, and represents the weight coefficients that sum to 1, Indicates Words The user feedback coefficient, The weight factor representing the user feedback coefficient; The sentence-based keywords , analyze and obtain key information of user sentences The specific steps are: replace three different sentence keywords Extract the sentence and get the corresponding verb, and then splice them in order to get the key information of the user sentence .

[0015] As a preferred technical solution of the present invention, the Words of The specific calculation expression is as follows: ; in, Indicates Words Same as the keyword set by the user, Indicates Words Different from the keyword set by the user.

[0016] The present invention also provides an intelligent question-answering method based on natural language processing, based on the above-mentioned intelligent question-answering system based on natural language processing, comprising the following steps: Step 1: Classify the information uploaded by the user according to the format of the information uploaded by the user to obtain the classification results: voice file, text file and image file; Step 2: Process the voice file and the image file according to the classification result to obtain a first input sentence, and the text file is directly used as the first input sentence; Step 3: Processing the first input sentence by the text input unit to generate a second input sentence; Step 4: After reading the second input sentence, execute the semantic extraction instruction to extract all key information of the second input sentence and output the sentence keywords , and execute information analysis instructions; Step 5: Based on sentence keywords , analyze and obtain key information of user sentences ; Step 6: Output the question and answer statements based on the entered question and answer statement database and the key information of the user statement, display them to the user, and wait for the user's feedback coefficient .

[0017] Compared with the prior art, the present invention provides an intelligent question-answering system and method based on natural language processing, which has the following beneficial effects: 1. The present invention uniformly processes voice files, text files and image files uploaded by users, performs preliminary information processing on the voice files or image files and enters them to obtain the first input sentence, and the text file is directly used as the first input sentence. The first input sentence is then processed by a text input unit to generate a second input sentence, and all key information of the second input sentence is extracted, sentence keywords are output, and user sentence key information is obtained by analysis. Question and answer sentences are output based on the question and answer sentence database that has been entered and the user sentence key information, thereby ensuring that the keywords of the user input content are fully extracted and the accuracy of the answer is ensured.

[0018] 2. The present invention can ensure that when keyword screening is performed, a large number of modifying terms or other irrelevant non-critical terms that appear in the process of user questions can be avoided to interfere with the keywords, thereby causing the keywords to appear repeatedly in non-critical parts such as sentences, resulting in fewer times during screening. After the user determines the main keywords, it can effectively ensure that the three selected keywords can answer the user's questions. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the system framework of the present invention; Figure 2 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] See also Figure 1-2 , an intelligent question-answering system based on natural language processing, including an information processing input module, a user intention analysis module, a question-answering sentence selection module, and a question-answering sentence feedback module; The information processing and input module includes a voice input unit, a text input unit and an image input unit. The information processing and input module identifies all information uploaded by the user, and classifies it according to the format of the information uploaded by the user to obtain the classification results: voice files, text files and image files, and allocates it to one of the voice input unit and the image input unit according to the classification results for information processing and input to obtain a first input sentence, and processes the first input sentence through the text input unit to generate a second input sentence, and uniformly processes the voice files, text files and image files uploaded by the user, performs preliminary information processing and input to obtain the first input sentence for the voice files or image files, and the text file is directly used as the first input sentence, thereby ensuring rapid information extraction of the input used; The specific formats of the voice files, text files and image files are shown in Table 1 below: Table 1

[0022] The specific steps for the voice input unit to generate an input sentence are as follows: Step A1: De-noise the voice file and divide the de-noised voice file into frames. The specific expression is as follows: ; in, Indicates the first Voice samples, The first samples, Indicates Window length; Step A2: After splitting the frame Voice samples Classification is performed and a first entry statement is generated.

[0023] The specific steps of the image input unit processing the image file to generate the first input sentence are as follows: Step B1: scale and grayscale the image file, convert the grayscale image into a black-and-white image, and extract the text area through threshold segmentation. The specific expression for converting the grayscale image into a black-and-white image and extracting the text area through threshold segmentation in step B1 is as follows: ; in, Represents pixel The qualified value of Represents pixel For the text area, Represents pixel The gray value of represents the pixel threshold; Step B2: Analyze the connectivity of image pixels and remove image noise; Step B3: Recognize the text after removing noise through the optical character recognition model, and arrange the recognition results in order to obtain the first input sentence. The specific steps of step B2 are as follows: Step B2.1: Select all the areas in the image that meet the requirements of step B1; Step B2.2: Comprehensively score the selected parts , comprehensive rating The specific calculation expression is as follows: ; in, and represents the weight coefficients that sum to 1, Indicates the length of the selected area. Indicates the width of the selection area. Indicates the area of ​​the selected area. Indicates the maximum area of ​​the selected area, which is the preset value for recognition; Step B2.3: Comprehensive scoring The parts that meet the judgment conditions are outputted and arranged from top to bottom according to the position of the frame selection center to obtain the first input sentence; The specific steps of the text input unit processing the first input sentence to generate the second input sentence are as follows: Step C1: read the connection word library in the question and answer sentence database, distinguish the nature of the words in the first input sentence, and remove prepositions and adjectives; Step C2: Split the first input sentence after elimination according to punctuation marks to obtain multiple segments, and use all the segments as the second input sentence.

[0024] The user intention analysis module includes a user information extraction unit and a user intention output unit. The user information extraction unit executes a semantic extraction instruction after reading the second input sentence, extracts all key information of the second input sentence, and outputs the sentence keywords. The user information extraction unit executes the semantic extraction instruction based on the second input sentence, and then processes the first input sentence through the text input unit to generate the second input sentence, and extracts all the key information of the second input sentence, outputs the sentence keywords, analyzes and obtains the user sentence key information, and outputs the question and answer sentence based on the input question and answer sentence database and the user sentence key information, thereby ensuring that the keywords of the user input content are fully extracted and the accuracy of the answer is ensured, and all the key information of the second input sentence is extracted and the sentence keywords are output The specific expression is as follows: ; in, Represents three different sentence keywords for output, and the acquisition steps are as follows: Step D1: extract all the words at the beginning of the second input sentence to form a first keyword library, extract all the words at the end of the second input sentence to form a second keyword library, and extract all the repeated words in the second input sentence to form a third keyword library; Step D2: Read the first keyword from the third keyword library Words , and read the Words Number of times in the first keyword library The number of times the second keyword library appears ; Step D3: Calculate the Words The key factor Until the traversal of the third keyword library is completed, the three with the largest key coefficients are selected as three different sentence keywords .

[0025] In step D3, the Words The key factor The expression is as follows: ; in, Indicates Words The number of times in the first keyword library, Indicates Words The number of times it appears in the second keyword library, and represents the weight coefficients that sum to 1, Indicates Words The user feedback coefficient, The weight factor representing the user feedback coefficient executes the information analysis instruction, which calls the user intention output unit and generates a user intention output based on the sentence keywords. , analyze and obtain key information of user sentences :Three different sentence keywords Extract the sentence and get the corresponding verb, and then splice them in order to get the key information of the user sentence .

[0026] The question and answer sentence selection module outputs question and answer sentences based on the entered question and answer sentence database and the key information of the user's sentence. This process matches the key information of the user's question with the stored question and answer sentence database. The question and answer sentence database is a data set containing multiple question and answer pairs. Usually these questions are common questions annotated manually or automatically. The system can find the most appropriate answer from the question and answer sentence database, show it to the user, and wait for the user's feedback coefficient , the question-answering sentence feedback module is based on the user feedback coefficient Output adjustment instructions. Before the next round of question and answer begins, adjust the user information extraction unit. Words of The specific calculation expression is as follows: ; in, Indicates Words Same as the keyword set by the user, Indicates Words Different from the keywords set by the user, by adding the means of user confirmation, it can be ensured that when performing keyword screening, the interference of too many modifying terms or other irrelevant non-critical terms on the keywords in the process of user questions can be avoided, thereby causing the keywords to appear repeatedly in non-critical parts such as sentences, resulting in fewer times during screening. After the user determines the main keywords, it can be effectively guaranteed that the three selected keywords can answer the user's questions.

[0027] The present invention also provides an intelligent question-answering method based on natural language processing, based on the above-mentioned intelligent question-answering system based on natural language processing, comprising the following steps: Step 1: Classify the information uploaded by the user according to the format of the information uploaded by the user to obtain the classification results: voice file, text file and image file; Step 2: Process the voice file and the image file according to the classification result to obtain a first input sentence, and the text file is directly used as the first input sentence; Step 3: Processing the first input sentence by the text input unit to generate a second input sentence; Step 4: After reading the second input sentence, execute the semantic extraction instruction to extract all key information of the second input sentence and output the sentence keywords , and execute information analysis instructions; Step 5: Based on sentence keywords , analyze and obtain key information of user sentences ; Step 6: Output the question and answer statements based on the entered question and answer statement database and the key information of the user statement, display them to the user, and wait for the user's feedback coefficient .

[0028] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. Intelligent question-answering system based on natural language processing, characterized by: It includes an information processing input module, a user intention analysis module, a question and answer sentence selection module, and a question and answer sentence feedback module; The information processing and input module includes a voice input unit, a text input unit and an image input unit. The information processing and input module identifies all information uploaded by the user, and classifies the information according to the format of the information uploaded by the user to obtain classification results: voice files, text files and image files, and assigns the information to one of the voice input unit and the image input unit according to the classification results for information processing and input to obtain a first input sentence, and processes the first input sentence through the text input unit to generate a second input sentence; The user intention analysis module includes a user information extraction unit and a user intention output unit. The user information extraction unit executes a semantic extraction instruction after reading the second input sentence, extracts all key information of the second input sentence, and outputs the sentence keywords. , and executes the information analysis instruction, the information analysis instruction calls the user intention output unit, and based on the sentence keywords , analyze and obtain key information of user sentences ; The question and answer sentence selection module outputs the question and answer sentence based on the entered question and answer sentence database and the key information of the user sentence, displays it to the user, and waits for the user's feedback coefficient The question-answering sentence feedback module is based on the user feedback coefficient Output an adjustment instruction, wherein the adjustment instruction adjusts the user information extraction unit before the next round of question and answer begins.

2. The intelligent question-answering system based on natural language processing according to claim 1, characterized in that: The specific steps of the voice input unit generating the input sentence are as follows: Step A1: De-noise the voice file and divide the de-noised voice file into frames. The specific expression is as follows: ； in, Indicates the first Voice samples, The first samples, Indicates Window length; Step A2: After splitting the frame Voice samples Classification is performed and a first entry statement is generated.

3. The intelligent question-answering system based on natural language processing according to claim 1, characterized in that: The specific steps of the image input unit processing the image file to generate the first input sentence are as follows: Step B1: scaling and graying the image file, converting the gray image into a black and white image, and extracting the text area by threshold segmentation; Step B2: Analyze the connectivity of image pixels and remove image noise; Step B3: Recognize the text after noise removal through an optical character recognition model, and arrange the recognition results in order to obtain a first input sentence.

4. The intelligent question-answering system based on natural language processing according to claim 3, characterized in that: In step B1, the grayscale image is converted into a black and white image, and the specific expression for extracting the text area by threshold segmentation is as follows: ； in, Represents pixel The qualified value of Represents pixel For the text area, Represents pixel The gray value of Represents the pixel threshold.

5. The intelligent question-answering system based on natural language processing according to claim 3, characterized in that: The specific steps of step B2 to mark the text blocks by analyzing the connectivity of the image pixels are as follows: Step B2.1: Select all the areas in the image that meet the requirements of step B1; Step B2.2: Comprehensively score the selected parts ; The overall score The specific calculation expression is as follows: ； in, and represents the weight coefficients that sum to 1, Indicates the length of the selected area. Indicates the width of the selection area. Indicates the area of ​​the selected area. Indicates the maximum area of ​​the selected area, which is the preset value for recognition; Step B2.3: Comprehensive scoring The part that meets the judgment condition is outputted and arranged from top to bottom according to the position of the frame selection center to obtain the first input statement.

6. The intelligent question-answering system based on natural language processing according to claim 1, characterized in that: The specific steps of the text input unit processing the first input sentence to generate the second input sentence are as follows: Step C1: read the connection word library in the question and answer sentence database, distinguish the nature of the words in the first input sentence, and remove prepositions and adjectives; Step C2: Split the first input sentence after elimination according to punctuation marks to obtain multiple segments, and use all the segments as the second input sentence.

7. The intelligent question-answering system based on natural language processing according to claim 6, characterized in that: The user information extraction unit executes a semantic extraction instruction based on the second input sentence, extracts all key information of the second input sentence, and outputs the sentence keywords. The specific expression is as follows: ； in, Represents three different sentence keywords for output, and the acquisition steps are as follows: Step D1: extract all the words at the beginning of the second input sentence to form a first keyword library, extract all the words at the end of the second input sentence to form a second keyword library, and extract all the repeated words in the second input sentence to form a third keyword library; Step D2: Read the first keyword from the third keyword library Words , and read the Words Number of times in the first keyword library The number of times the second keyword library appears ; Step D3: Calculate the Words The key factor Until the third keyword library is traversed, the three with the largest key coefficients are selected as three different sentence keywords. .

8. The intelligent question-answering system based on natural language processing according to claim 7, characterized in that: In step D3, the calculation Words The key factor The expression is as follows: ； in, Indicates Words The number of times in the first keyword library, Indicates Words The number of times it appears in the second keyword library, and represents the weight coefficients that sum to 1, Indicates Words The user feedback coefficient, The weight factor representing the user feedback coefficient; The sentence-based keywords , analyze and obtain key information of user sentences The specific steps are: replace three different sentence keywords Extract the sentence and get the corresponding verb, and then splice them in order to get the key information of the user sentence .

9. The intelligent question-answering system based on natural language processing according to claim 8, characterized in that: The said Words of The specific calculation expression is as follows: ； in, Indicates Words Same as the keyword set by the user, Indicates Words Different from the keyword set by the user.

10. An intelligent question-answering method based on natural language processing, applied to the intelligent question-answering system based on natural language processing according to any one of claims 1 to 9, characterized in that: The following steps are involved: Step 1: Classify the information uploaded by the user according to the format of the information uploaded by the user to obtain the classification results: voice file, text file and image file; Step 2: Process the voice file and the image file according to the classification result to obtain a first input sentence, and the text file is directly used as the first input sentence; Step 3: Processing the first input sentence by the text input unit to generate a second input sentence; Step 4: After reading the second input sentence, execute the semantic extraction instruction to extract all key information of the second input sentence and output the sentence keywords , and execute information analysis instructions; Step 5: Based on sentence keywords , analyze and obtain key information of user sentences ; Step 6: Output the question and answer statements based on the entered question and answer statement database and the key information of the user statement, display them to the user, and wait for the user's feedback coefficient .

Images