An intelligent question-answering system based on data analysis

Abstract

The present invention discloses an intelligent question-answering system based on data analysis, comprising a data collection and integration module, a data analysis and modeling module, a model library, a user import module, a data conversion module, a data analysis module, a data matching module, and a data export module; the data collection and integration module is used to determine the data source, collect data, and integrate data; the data analysis and modeling module is used to perform data analysis and model construction; the model library is used to store the models constructed in the data analysis and modeling module; the user import module is used for users who need to perform intelligent question-answering to import the content of the required questions and answers; the data conversion module is used to convert the content of the user-imported required questions and answers and obtain converted content information; the data analysis module is used to analyze the converted content information and obtain keyword information. The present invention can perform more accurate and rapid intelligent question-answering.

Description

Technical Field

[0001] The present invention relates to the field of question-answering systems, and in particular to an intelligent question-answering system based on data analysis. Background Art

[0002] Question-answering (Q&A) systems are designed to automatically answer questions posed by users in natural language. These systems work by understanding questions, analyzing information, and retrieving or generating answers from structured or unstructured data sources. Q&A systems are widely used in search engines, intelligent customer service, education, healthcare, law, and other fields to improve the efficiency and accuracy of information acquisition.

[0003] The existing question-answering system has problems such as slow response speed and low accuracy, which has a certain impact on the use of the question-answering system. Therefore, an intelligent question-answering system based on data analysis is proposed. Summary of the Invention

[0004] The technical problem to be solved by the present invention is: how to solve the problem of the existing question-answering system that the question-answering speed is not fast enough and the accuracy is not high enough, and provide an intelligent question-answering system based on data analysis.

[0005] The present invention solves the above technical problems through the following technical solutions, which include a data collection and integration module, a data analysis and modeling module, a model library, a user import module, a data conversion module, a data analysis module, a data matching module and a data export module;

[0006] The data collection and integration module is used to determine the data source, collect data and integrate data;

[0007] The data analysis and modeling module is used to perform data analysis and model building;

[0008] The model library is used to store the models constructed in the data analysis and modeling module;

[0009] The user import module is used for users who need to conduct intelligent question and answer to import the required question and answer content;

[0010] The data conversion module is used to convert the content of the questions and answers required by the user and obtain the converted content information;

[0011] The data analysis module is used to analyze the converted content information and obtain keyword information;

[0012] The data matching module is used to import keyword information into the model library and retrieve the question-answering model matching the keyword from the model library;

[0013] The data export module is used to export the matched question-answering model.

[0014] Furthermore, the specific process of the data collection and integration module for determining the data source, collecting data, and integrating data is as follows:

[0015] Determine the data source and clarify the types of data that need to be collected, including historical disaster data, emergency resource information, on-site monitoring data, and social media public opinion;

[0016] Data collection: using crawler technology, API interfaces or database synchronization to collect relevant data of the required data types from various data sources;

[0017] Data integration: pre-process the collected relevant data and import it into the database.

[0018] Furthermore, the specific process of data analysis and model building is as follows:

[0019] Use descriptive and inferential statistics to reveal inherent patterns and trends in data;

[0020] Automatically or semi-automatically discover patterns, associations, and clusters in data through algorithms to identify hidden information or predict future trends;

[0021] Use machine learning algorithms to enable computers to learn from data and improve their performance to obtain data analysis results:

[0022] Based on the results of data analysis, select the model type that matches the data analysis results;

[0023] Depending on the selected model type, mathematical languages ​​or programming tools are used for modeling;

[0024] After that, the model is verified by historical data or simulation data. After passing the verification, the model is imported into the model library.

[0025] Furthermore, the specific process of obtaining the conversion content information is as follows:

[0026] Extract the content of the question and answer required by the user. When the content of the question and answer required by the user is text content, directly export it as converted content information;

[0027] When the content of the question and answer that the user imports is voice content, the voice content is recognized and processed to convert the text information into converted content information;

[0028] When the content of the required question imported by the user is a picture, text recognition processing is performed on the picture to obtain converted text information, and then the converted text information is converted into converted content information.

[0029] Furthermore, in the speech-to-text recognition process of the speech content, an algorithm based on a statistical model, an algorithm based on a neural network, and an algorithm based on deep learning are applied simultaneously to perform speech-to-text recognition;

[0030] Marking the text content obtained by the algorithm based on the statistical model as the first content;

[0031] Marking the text content obtained by the neural network-based algorithm as second content;

[0032] Marking text content obtained through a deep learning-based algorithm as third content;

[0033] Comparing the obtained first content, second content, and third content, and when the first content, second content, and third content are all the same, exporting any one of the first content, second content, and third content as converted text information;

[0034] When any two of the first content, the second content, and the third content are identical, and the other one is different, any one of the two identical contents is exported as converted text information;

[0035] When any two of the first content, the second content, and the third content are different, re-conversion information is generated.

[0036] Furthermore, the data analysis module is used to analyze the converted content information, and the specific process of obtaining keyword information is as follows:

[0037] First, perform data preprocessing on the converted text information to remove HTML tags, special characters, stop words and punctuation marks in the text;

[0038] Then perform word segmentation to divide the text into meaningful vocabulary units;

[0039] Perform lemmatization to restore vocabulary to its basic form;

[0040] Then perform feature extraction, using statistical methods, topic model-based methods and deep learning models to extract vocabulary features and match keywords;

[0041] Perform keyword filtering and further filter keywords based on business needs or specific rules;

[0042] At the same time, keyword merging is performed. For keywords with similar semantics, they are merged into a more general keyword to obtain the required keyword information.

[0043] Furthermore, the specific process of retrieving the question-answering model matching the keyword from the model library is as follows: importing the acquired keyword information into the model library;

[0044] First retrieve all models matching the keyword information from the model library;

[0045] Then, all models matching the keyword information are sorted in descending order according to the number of matching keyword information;

[0046] Finally, the model with the largest number of matched keyword information is exported as the question-answering model for the first question-answer response;

[0047] The model with the second largest number of matched keyword information is exported as the question-answering model for the second question-answering response;

[0048] Similarly, the model with at least the xth largest number of matched keyword information is exported as the question-answering model for the xth question-answering response, where x is a positive integer.

[0049] Furthermore, the intelligent question-answering system also evaluates the relevant information of each question-answering reply content, and processes the relevant information of each question-answering reply content to obtain question-answering evaluation information.

[0050] Furthermore, the question-and-answer evaluation information includes whether the evaluation is passed or not. The process of obtaining the question-and-answer evaluation information is as follows:

[0051] Extract relevant information about the response content of each question and answer session, including the time when the user asked the question, the time when the system responded, and the user's score after the question and answer session;

[0052] The time point when the user asks the question is marked as T, and the time point when the system responds is marked as Y. The user's score after the question and answer session includes first-category score, second-category score, and third-category score. The specific value of the third-category score is greater than the specific value of the second-category score and greater than the specific value of the first-category score. The score after the question and answer session is marked as K;

[0053] First, process T and Y, calculate the difference between the system response time Y and the user question time T for each question, and obtain the single response time difference Yt;

[0054] Then calculate the average of the single response time difference Yt of all questions and answers to obtain the time evaluation parameter Ytt;

[0055] Then, the time evaluation parameter Ytt is assigned a correction value G1, and the score K after the question and answer session is assigned a correction value G2, G1+G2=1, G1>G2;

[0056] Through the formula Ytt*G1+K*G2=Yk, the comprehensive evaluation parameter Yk is obtained;

[0057] When the comprehensive evaluation parameter Y is greater than or equal to the preset value, the evaluation is generated as failed;

[0058] When the comprehensive evaluation parameter Y is less than the preset value, the evaluation is passed.

[0059] Compared with the existing technology, the present invention has the following advantages: the intelligent question-answering system based on data analysis can ensure that the collected data is reliable, accurate and meets business needs by clarifying the data source. This helps to reduce analysis errors and decision-making errors caused by data quality problems, automatically extracts data from multiple data sources, greatly improves the efficiency and speed of data collection, and through comprehensive data collection, can ensure that the system obtains as much relevant information as possible, providing a solid foundation for subsequent data analysis; at the same time, users can input different types of question content in a variety of different forms according to actual needs, thereby meeting the different usage needs of users, and finally conduct a comprehensive evaluation of the question-answering process, which can understand the question-answering efficiency and accuracy of the question-answering system, so as to more accurately perform relevant optimization to improve the question-answering experience, so that the system can perform intelligent question-answering with higher accuracy and faster speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a system block diagram of the present invention. DETAILED DESCRIPTION

[0061] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process. However, the protection scope of the present invention is not limited to the following embodiment.

[0062] like Figure 1 As shown, this embodiment provides a technical solution: an intelligent question-answering system based on data analysis, including a data collection and integration module, a data analysis and modeling module, a model library, a user import module, a data conversion module, a data analysis module, a data matching module and a data export module;

[0063] The data collection and integration module is used to determine the data source, collect data and integrate data;

[0064] The data analysis and modeling module is used to perform data analysis and model building;

[0065] The model library is used to store the models constructed in the data analysis and modeling module;

[0066] The user import module is used for users who need to conduct intelligent question and answer to import the required question and answer content;

[0067] The data conversion module is used to convert the content of the questions and answers required by the user and obtain the converted content information;

[0068] The data analysis module is used to analyze the converted content information and obtain keyword information;

[0069] The data matching module is used to import keyword information into the model library and retrieve the question-answering model matching the keyword from the model library;

[0070] The data export module is used to export the matched question-answering model.

[0071] Data source determination: First, clarify the type of data that needs to be collected, including historical disaster data, emergency resource information, on-site monitoring data, and social media public opinion.

[0072] Data collection: Use crawler technology, API interface, database synchronization and other methods to collect required data from various data sources.

[0073] Data integration: pre-process the collected relevant data and import it into the database.

[0074] The specific process of data analysis and model construction is as follows:

[0075] Use descriptive and inferential statistics to reveal inherent patterns and trends in data;

[0076] Automatically or semi-automatically discover patterns, associations, and clusters in data through algorithms to identify hidden information or predict future trends;

[0077] Use machine learning algorithms to enable computers to learn from data and improve their performance to obtain data analysis results:

[0078] Based on the results of data analysis, select the model type that matches the data analysis results;

[0079] Depending on the selected model type, mathematical languages ​​or programming tools are used for modeling;

[0080] After that, the model is verified by historical data or simulation data. After passing the verification, the model is imported into the model library;

[0081] Through the above process, we can obtain relevant models that are more in line with the standards. Only by establishing enough models can we ensure that when subsequent users ask questions related to emergency plans, we can provide users with relevant plans quickly and accurately.

[0082] The specific process of obtaining the conversion content information is as follows:

[0083] Extract the content of the question and answer required by the user. When the content of the question and answer required by the user is text content, directly export it as converted content information;

[0084] When the content of the question and answer that the user imports is voice content, the voice content is recognized and processed to convert the text information into converted content information;

[0085] When the content of the question imported by the user is a picture, the picture is subjected to text recognition processing to obtain converted text information, and then the converted text information is converted into converted content information;

[0086] The above process improves the user experience, and the diversified input methods allow users to choose the most appropriate input method according to their preferences and current environment. Whether it is fast typing, taking photos and uploading pictures, or direct voice input, they can get a quick response.

[0087] It is more convenient. For users with poor eyesight, hand problems or who are on the move, the voice input and picture-to-text functions are particularly convenient, reducing input barriers.

[0088] Immediacy: Voice input is particularly suitable for scenarios where you need to quickly record or ask questions;

[0089] The system's flexibility has been enhanced to adapt to different scenarios. Different input methods are suitable for different scenarios and needs. The system provides multiple input methods to better meet the diverse needs of different users.

[0090] Improve the accuracy of questions and answers: Reduce input errors: The speech-to-text and image-to-text functions use automatic recognition and conversion technology to reduce spelling errors or misunderstandings that may occur when users enter data manually.

[0091] Accurate recognition: Advanced OCR (optical character recognition) and speech recognition technologies can accurately identify text in images and speech content, providing a more accurate data foundation for subsequent question-and-answer processing.

[0092] Promote barrier-free communication: Support users with special needs: For users with special needs such as visual impairment and hearing impairment, these input methods provide a more friendly way of communication and promote barrier-free transmission of information.

[0093] In the speech-to-text recognition process, algorithms based on statistical models, algorithms based on neural networks, and algorithms based on deep learning are applied to simultaneously perform speech-to-text recognition.

[0094] Marking the text content obtained by the algorithm based on the statistical model as the first content;

[0095] Marking the text content obtained by the neural network-based algorithm as second content;

[0096] Marking text content obtained through a deep learning-based algorithm as third content;

[0097] Comparing the obtained first content, second content, and third content, and when the first content, second content, and third content are all the same, exporting any one of the first content, second content, and third content as converted text information;

[0098] When any two of the first content, the second content, and the third content are identical, and the other one is different, any one of the two identical contents is exported as converted text information;

[0099] When any two of the first content, the second content, and the third content are different, re-conversion information is generated;

[0100] Through the above process, the accuracy of the converted text content can be guaranteed, thereby improving the accuracy of the final question and answer response content.

[0101] The data analysis module is used to analyze the converted content information. The specific process of obtaining keyword information is as follows:

[0102] First, perform data preprocessing on the converted text information to remove HTML tags, special characters, stop words and punctuation marks in the text;

[0103] Then perform word segmentation to divide the text into meaningful vocabulary units;

[0104] Perform lemmatization to restore vocabulary to its basic form;

[0105] Then perform feature extraction, using statistical methods, topic model-based methods and deep learning models to extract vocabulary features and match keywords;

[0106] Perform keyword filtering and further filter keywords based on business needs or specific rules;

[0107] At the same time, keywords are merged. For keywords with similar semantics, they are merged into a more general keyword to obtain the required keyword information;

[0108] Through the above process, keyword information can be obtained more accurately, thereby improving the accuracy of the final question and answer response content.

[0109] The specific process of retrieving the question-answering model matching the keyword from the model library is as follows: importing the acquired keyword information into the model library;

[0110] First retrieve all models matching the keyword information from the model library;

[0111] Then, all models matching the keyword information are sorted in descending order according to the number of matching keyword information;

[0112] Finally, the model with the largest number of matched keyword information is exported as the question-answering model for the first question-answer response;

[0113] The model with the second largest number of matched keyword information is exported as the question-answering model for the second question-answering response;

[0114] Similarly, the model with at least the xth largest number of matched keyword information is exported as the question-answering model for the xth question-answering response, where x is a positive integer.

[0115] The intelligent question-answering system also evaluates and processes the relevant information of each question-answering response to obtain question-answering evaluation information.

[0116] Question and answer evaluation information includes whether the evaluation is passed or failed. The process of obtaining question and answer evaluation information is as follows:

[0117] Extract relevant information about the response content of each question and answer session, including the time when the user asked the question, the time when the system responded, and the user's score after the question and answer session;

[0118] The time point when the user asks the question is marked as T, and the time point when the system responds is marked as Y. The user's score after the question and answer session includes first-category score, second-category score, and third-category score. The specific value of the third-category score is greater than the specific value of the second-category score and greater than the specific value of the first-category score. The score after the question and answer session is marked as K;

[0119] First, process T and Y, calculate the difference between the system response time Y and the user question time T for each question, and obtain the single response time difference Yt;

[0120] Then calculate the average of the single response time difference Yt of all questions and answers to obtain the time evaluation parameter Ytt;

[0121] Then, the time evaluation parameter Ytt is assigned a correction value G1, and the score K after the question and answer session is assigned a correction value G2, G1+G2=1, G1>G2;

[0122] Through the formula Ytt*G1+K*G2=Yk, the comprehensive evaluation parameter Yk is obtained;

[0123] When the comprehensive evaluation parameter Y is greater than or equal to the preset value, the evaluation is generated as failed;

[0124] When the comprehensive evaluation parameter Y is less than the preset value, the evaluation is passed.

[0125] Failure to pass the assessment means that the question-answering system needs to be optimized to improve the quality of question-answer responses.

[0126] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0127] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0128] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. An intelligent question-answering system based on data analysis, characterized in that: It includes data collection and integration module, data analysis and modeling module, model library, user import module, data conversion module, data analysis module, data matching module and data export module; The data collection and integration module is used to determine the data source, collect data and integrate data; The data analysis and modeling module is used to perform data analysis and model building; The model library is used to store the models constructed in the data analysis and modeling module; The user import module is used for users who need to conduct intelligent question and answer to import the required question and answer content; The data conversion module is used to convert the content of the questions and answers required by the user and obtain the converted content information; The data analysis module is used to analyze the converted content information and obtain keyword information; The data matching module is used to import keyword information into the model library and retrieve the question-answering model matching the keyword from the model library; The data export module is used to export the matched question-answering model; The specific process of data source determination, data collection and data integration by the data collection and integration module is as follows: Determine the data source and clarify the types of data that need to be collected, including historical disaster data, emergency resource information, on-site monitoring data, and social media public opinion; Data collection: using crawler technology, API interfaces or database synchronization to collect relevant data of the required data types from various data sources; Data integration: pre-process the collected relevant data and import it into the database; The specific process of data analysis and model construction is as follows: Use descriptive and inferential statistics to reveal inherent patterns and trends in data; Automatically or semi-automatically discover patterns, associations, and clusters in data through pre-set algorithms to identify hidden information or predict future trends; Use machine learning algorithms to enable computers to learn from data and improve their performance to obtain data analysis results: Based on the results of data analysis, select the model type that matches the data analysis results; Depending on the selected model type, mathematical languages ​​or programming tools are used for modeling; Then the model is validated; Question and answer evaluation information includes whether the evaluation is passed or failed. The process of obtaining question and answer evaluation information is as follows: Extract relevant information about the response content of each question and answer session, including the time when the user asked the question, the time when the system responded, and the user's score after the question and answer session; The time point when the user asks the question is marked as T, and the time point when the system responds is marked as Y. The user's score after the question and answer session includes first-category score, second-category score, and third-category score. The specific value of the third-category score is greater than the specific value of the second-category score and greater than the specific value of the first-category score. The score after the question and answer session is marked as K; First, process T and Y, calculate the difference between the system response time Y and the user question time T for each question, and obtain the single response time difference Yt; Then calculate the average of the single response time difference Yt of all questions and answers to obtain the time evaluation parameter Ytt; Then, the time evaluation parameter Ytt is assigned a correction value G1, and the score K after the question and answer session is assigned a correction value G2, G1+G2=1, G1>G2; Through the formula Ytt*G1+K*G2=Yk, the comprehensive evaluation parameter Yk is obtained; When the comprehensive evaluation parameter Y is greater than or equal to the preset value, the evaluation is generated as failed; When the comprehensive evaluation parameter Y is less than the preset value, the evaluation is passed, and the model is verified through historical data or simulation data. After the verification is passed, the model is imported into the model library.

2. The intelligent question-answering system based on data analysis according to claim 1, characterized in that: The specific process of obtaining the conversion content information is as follows: Extract the content of the question and answer required by the user. When the content of the question and answer required by the user is text content, directly export it as converted content information; When the content of the question and answer that the user imports is voice content, the voice content is recognized and processed to convert the text information into converted content information; When the content of the required question imported by the user is a picture, text recognition processing is performed on the picture to obtain converted text information, and then the converted text information is converted into converted content information.

3. The intelligent question-answering system based on data analysis according to claim 2, characterized in that: During the speech-to-text recognition process, algorithms based on statistical models, neural networks, and deep learning are used to simultaneously perform speech-to-text recognition. Marking the text content obtained by the algorithm based on the statistical model as the first content; Marking the text content obtained by the neural network-based algorithm as second content; Marking text content obtained through a deep learning-based algorithm as third content; Comparing the obtained first content, second content, and third content, and when the first content, second content, and third content are all the same, exporting any one of the first content, second content, and third content as converted text information; When any two of the first content, the second content, and the third content are identical, and the other one is different, any one of the two identical contents is exported as converted text information; When any two of the first content, the second content, and the third content are different, re-conversion information is generated.

4. The intelligent question-answering system based on data analysis according to claim 1, characterized in that: The data analysis module is used to analyze the converted content information. The specific process of obtaining keyword information is as follows: First, perform data preprocessing on the converted text information to remove HTML tags, special characters, stop words and punctuation marks in the text; Then perform word segmentation to divide the text into meaningful vocabulary units; Perform lemmatization to restore vocabulary to its basic form; Then perform feature extraction, using statistical methods, topic model-based methods and deep learning models to extract vocabulary features and match keywords; Perform keyword filtering and further filter keywords based on business needs or specific rules; At the same time, keyword merging is performed. For keywords with similar semantics, they are merged into a more general keyword to obtain the required keyword information.

5. The intelligent question-answering system based on data analysis according to claim 1, characterized in that: The specific process of retrieving the question-answering model matching the keyword from the model library is as follows: importing the acquired keyword information into the model library; First retrieve all models matching the keyword information from the model library; Then, all models matching the keyword information are sorted in descending order according to the number of matching keyword information; Finally, the model with the largest number of matched keyword information is exported as the question-answering model for the first question-answer response; The model with the second largest number of matched keyword information is exported as the question-answering model for the second question-answering response; Similarly, the model with at least the xth largest number of matched keyword information is exported as the question-answering model for the xth question-answering response, where x is a positive integer.

6. The intelligent question-answering system based on data analysis according to claim 1, characterized in that: The intelligent question-answering system also evaluates and processes the relevant information of each question-answering response to obtain question-answering evaluation information.

Images