Data processing method and system for question and answer platform

Abstract

The invention belongs to the technical field of text data processing, and particularly relates to a data processing method and system for a question and answer platform. The method comprises the following steps: firstly, collecting and preprocessing questioning and replying data from different regional sub-regions, and constructing a to-be-analyzed data set; thirdly, accurately identifying invalid response features in the reply data by applying a BERT model and combining a supervised learning technology; then, performing clustering analysis on question data associated with the invalid response by adopting an LDA topic model, extracting a keyword feature set, and verifying whether the keyword feature set meets a preset classification standard or not; and if not, iteratively updating the keyword feature set until the keyword feature set reaches the standard. And finally, aiming at the keyword feature set meeting the condition, determining the question feature with the highest priority, and deeply extracting the reason and action feature items behind the question by means of a BERT-LSTM-CRF model. According to the method, the question identification and analysis capability of the question and answer platform is remarkably improved, and powerful support is provided for quick response and high-quality service.

Description

Technical Field

[0001] The present invention belongs to the technical field of text data processing, and in particular relates to a data processing method and system for a question-and-answer platform. Background Art

[0002] With the rapid development of Internet technology, online question-and-answer platforms have become an important way for people to acquire knowledge and solve problems. These platforms provide the public with a convenient way to interact with information by collecting user questions and response data. In the prior art, traditional text analysis methods, such as keyword extraction and sentiment analysis, are usually used to process data on online question-and-answer platforms to mine user needs and feedback.

[0003] The existing technology has the following main defects when processing online question-and-answer platform data:

[0004] First, there is insufficient evaluation of response effectiveness, making it difficult for platforms to accurately identify which responses truly solve user problems, resulting in a damaged user experience and a waste of resources.

[0005] Secondly, problem feature extraction is inaccurate. Traditional text analysis methods, such as rule-based keyword extraction, often cannot accurately capture the core information of the problem, which limits the platform's in-depth understanding of user needs and also affects the effectiveness of subsequent problem governance activities.

[0006] Finally, existing technologies lack consideration of cross-regional spatial differences and ignore the significant differences in user needs and problem characteristics in different regional spaces. As a result, the platform is unable to adapt to local conditions and implement precise policies when handling user questions, which reduces the platform's response speed and service quality, and may cause user dissatisfaction and loss. Summary of the invention

[0007] The purpose of the present invention is to provide a data processing method and system for a question-and-answer platform to solve the problems raised in the background technology.

[0008] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0009] In a first aspect, the present invention proposes a data processing method for a question-and-answer platform, the method comprising:

[0010] Obtaining question data of questioners in different sub-regions based on the regional spatial division and reply data of responders from the online question-answering platform and preprocessing them to form a data set to be analyzed; wherein the question data includes at least two question features;

[0011] The BERT model is used to classify the response data in the data set to be analyzed, and the characteristics of invalid responses of the responding party are identified by combining the supervised learning algorithm;

[0012] Using the LDA topic model to perform topic clustering on the question data corresponding to the invalid response features to obtain a keyword feature set, and determining whether the keyword feature set meets a preset classification condition;

[0013] If the classification condition is not met, updating the keyword feature set until it meets the condition;

[0014] If the classification conditions are met, the problem feature with the highest priority in the keyword feature set is determined, and the cause feature item and action feature item corresponding to the problem feature are extracted from the invalid response feature based on the BERT-LSTM-CRF model.

[0015] Furthermore, the invalid response feature is specifically: according to the degree to which the response data solves the problem characteristics of the questioner, the effectiveness of the response data is divided into four categories: solved problems, promised problems, problems being solved, and unresolved problems. The response to unresolved problems is the invalid response feature.

[0016] Furthermore, the BERT model is used to classify the response data in the data set to be analyzed, and the characteristics of invalid responses of the responding party are identified in combination with the supervised learning algorithm, including:

[0017] Manual annotation to obtain the initial training samples;

[0018] Compare the coding results of the two annotators and discuss and confirm the samples with inconsistent annotations;

[0019] Divide the data into training set, validation set and test set according to the preset ratio;

[0020] The BERT model is used for training and final prediction, and the classification results are evaluated using three indicators: recall, precision, and F1-score.

[0021] Furthermore, the LDA topic model is used to perform topic clustering on the question data corresponding to the invalid response features to obtain a keyword feature set, including:

[0022] Determining the optimal number of question topics in the question data based on the topic perplexity;

[0023] Through manual inspection, similar question topics are merged, and the final number of question topics and the invalid response features corresponding to each question topic are determined to form a keyword feature set.

[0024] Furthermore, after the question data corresponding to the invalid response features are clustered by topic using the LDA topic model, the method further includes:

[0025] Analyzing the distribution of different problem features in different sub-areas based on the spatial distribution characteristics of the keyword feature set; and,

[0026] According to the amount of data related to different question themes in different sub-regions in the question data, the attention trends of questioners in different sub-regions on different question themes are evaluated.

[0027] Furthermore, the determining whether the keyword feature set meets a preset classification condition includes:

[0028] Determine at least two question topics based on the keyword feature set, and keywords with a predetermined ranking in importance for each question topic;

[0029] For each piece of question data, counting the number of keywords in the question data that are included in the question data and belong to a preset ranking of each question topic;

[0030] If the question data contains at least three keywords from the same topic question with a preset ranking, the question data is determined to be related to the question topic and meets the classification condition; otherwise, it does not meet the classification condition.

[0031] Furthermore, if the classification condition is met, determining the question feature with the highest priority in the keyword feature set includes:

[0032] Determine the priority criteria for different question topics based on the distribution of question topics in different sub-regions and the attention trends of questioners in different sub-regions on different question topics;

[0033] The question feature with the highest priority in the keyword feature set is determined based on the priority standard.

[0034] Furthermore, the BERT-LSTM-CRF model is used to extract the cause feature item and the action feature item corresponding to the problem feature from the invalid response feature, including:

[0035] Features are extracted from the invalid response features, wherein BERT is used for text encoding, LSTM is used for sequence modeling, and CRF is used for sequence labeling, so as to identify and extract the invalid response features, and further parse the cause feature items and action feature items related to the problem features from the invalid response features.

[0036] Furthermore, after extracting the cause feature item and the action feature item corresponding to the problem feature from the invalid response feature based on the BERT-LSTM-CRF model, it also includes:

[0037] Based on BETR, the cause feature items and action feature items are represented by vectors, the extracted cause feature items and action feature items are clustered using the K-Means model, the distance between two points is measured using cosine similarity, and the optimal number of topics is determined by judging the sum of the distances from all points in each category to the cluster center point. After further manual merging and screening, the final cause feature items and action feature items are obtained;

[0038] According to the clustering results and preset evaluation criteria, the importance of each cause characteristic item is evaluated, and the priority of the corresponding problem theme governance activities is determined based on the importance.

[0039] In a second aspect, the present invention proposes a data processing system for a question-and-answer platform, the system comprising:

[0040] A data acquisition module is used to acquire question data of questioners in different sub-regions divided based on geographical space and reply data of responders from the online question-answering platform and perform preprocessing to form a data set to be analyzed; wherein the question data includes at least two question features;

[0041] The first analysis module is used to classify the response data in the data set to be analyzed by using the BERT model, and identify the invalid response characteristics of the responding party in combination with the supervised learning algorithm;

[0042] The second analysis module is used to perform topic clustering on the question data corresponding to the invalid response features using an LDA topic model to obtain a keyword feature set, and determine whether the keyword feature set meets a preset classification condition;

[0043] A data updating module, used for updating the keyword feature set until it meets the classification condition when the classification condition is not met;

[0044] The data processing module is used to determine the problem feature with the highest priority in the keyword feature set when the classification conditions are met, and extract the cause feature item and action feature item corresponding to the problem feature from the invalid response feature based on the BERT-LSTM-CRF model.

[0045] The beneficial effects of the present invention are:

[0046] 1. The present invention classifies the reply data of the online question-and-answer platform by introducing the BERT model, effectively identifies the invalid response features, and then performs thematic clustering on the question data to extract the highest priority question features. In this process, the deep learning ability of the BERT model ensures the accuracy and efficiency of response classification, enabling the platform to quickly identify and solve user problems. At the same time, the accurate identification of invalid responses helps the platform to intervene in time, reduce the interference of invalid responses, and improve the effectiveness of the overall response and user satisfaction. In addition, by extracting the highest priority question features, the platform can more accurately locate user needs, provide users with more personalized services, and further enhance the user experience.

[0047] 2. The present invention uses the BERT-LSTM-CRF model to extract the cause feature items and action feature items of the corresponding problem features from the invalid responses, providing rich data support for the platform. Through the analysis and evaluation of these feature items, the platform can deeply understand the root causes and influencing factors of user problems, so as to formulate more accurate and effective problem governance strategies. At the same time, combined with the consideration of cross-regional spatial differences, the platform can formulate differentiated service solutions based on user needs and problem characteristics in different regional spaces, and realize the optimal allocation and efficient use of resources. This not only improves the response speed and service quality of the platform, but also helps to reduce operating costs and achieve sustainable development. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flowchart of a data processing method for a question-and-answer platform provided in an embodiment of the present application;

[0049] Figure 2 A schematic diagram of the process for implementing the classification conditions in the embodiments of the present application;

[0050] Figure 3 A technical roadmap for a data processing system of a question-and-answer platform provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The present application is further described in detail below in conjunction with the accompanying drawings. It is necessary to point out here that the following specific implementation methods are only used to further illustrate the present application and cannot be understood as limiting the scope of protection of the present application. Technical personnel in this field can make some non-essential improvements and adjustments to the present application based on the above application content.

[0052] Example 1

[0053] like Figure 1 As shown, this embodiment proposes a data processing method for a question-and-answer platform, and the method comprises the following steps:

[0054] S1. Obtain the question data of questioners in different sub-regions based on geographical space division and the reply data of responders from the online question-and-answer platform and pre-process them to form a data set to be analyzed; wherein the question data includes at least two question features; the pre-processing includes: removing special symbols and stop words in the question data and reply data, and performing word segmentation to form the data set to be analyzed.

[0055] S2. The BERT model (Bidirectional Encoder Representations from Transformers model is a pre-trained model based on Transformer, mainly used for language representation) is used to classify the response data in the data set to be analyzed, and the supervised learning algorithm is used to identify the invalid response features of the respondent. Specifically, the invalid response features are: according to the degree to which the response data solves the problem characteristics of the questioner, the effectiveness of the response data is divided into four categories: solved problems, promised problems, problems being solved, and unresolved problems. The response to unresolved problems is an invalid response feature.

[0056] S3. Use the LDA (Latent Dirichlet Allocation) topic model to perform topic clustering on the question data corresponding to the invalid response features to obtain a keyword feature set, and determine whether the keyword feature set meets the preset classification conditions;

[0057] S4. If the classification condition is not met, update the keyword feature set until it meets the condition;

[0058] S5. If the classification conditions are met, the problem feature with the highest priority in the keyword feature set is determined, and the cause feature item and action feature item corresponding to the problem feature are extracted from the invalid response feature based on the BERT-LSTM-CRF model.

[0059] It can be understood that this embodiment realizes in-depth mining and analysis of online question-and-answer platform data through five core steps. Step 1, data acquisition and preprocessing, ensures the accuracy and availability of the data, and lays a solid foundation for subsequent analysis. Step 2, using the BERT model to classify the reply data, accurately identify invalid responses, and effectively improve the effectiveness of the platform response and user satisfaction. Step 3, using the LDA topic model to perform topic clustering on the question data corresponding to invalid responses, revealing hot issues that users are concerned about, and providing valuable market insights for the platform. Step 4, determination and updating of keyword feature sets ensures the accuracy and timeliness of the analysis results, and provides strong support for platform decision-making. Step 5, through the BERT-LSTM-CRF model, the causes and action items of the problem features are extracted, and the essence of the user's problem is deeply analyzed, providing a scientific basis for the platform to optimize services and improve user experience.

[0060] It should be noted that the BERT-LSTM-CRF model of the present invention is used to extract concept phrases. This model uses BERT to encode the text to obtain the vector representation of the text and input it into the LSTMCRF layer to predict the label of each character in the sentence. In addition, common NER models such as hidden Markov model, CRF, bidirectional LSTM (Bi-LSTM) and Bi-LSTM-CRF are used for comparative analysis.

[0061] Further preferably, in step 2, the BERT model is used to classify the response data in the data set to be analyzed, and the invalid response features of the responding party are identified in combination with the supervised learning algorithm, including: manually labeling to obtain the initial training samples; comparing the encoding results of the two labelers, and discussing and determining the samples with inconsistent labeling; dividing the data into training set, validation set and test set according to a preset ratio; using the BERT model for training and final prediction, and using the three indicators of recall rate, precision rate and F1-score to evaluate the classification results.

[0062] Further preferably, in step 3, the LDA topic model is used to perform topic clustering on the question data corresponding to the invalid response features to obtain a keyword feature set, including: determining the optimal number of question topics in the question data based on the topic perplexity; merging similar question topics through manual inspection, and determining the final number of question topics and the invalid response features corresponding to each question topic to form a keyword feature set.

[0063] Further preferably, after the question data corresponding to the invalid response features are clustered by the LDA topic model, the method further includes: analyzing the distribution of different question features in different sub-regions based on the spatial distribution characteristics of the keyword feature set; and evaluating the attention trends of questioners in different sub-regions to different question topics based on the number of data related to different question topics in different sub-regions in the question data.

[0064] After topic clustering, by analyzing the distribution of different problem characteristics in different sub-regions, we can clearly see the hot issues that users in different regions are concerned about and their differences. This helps the platform understand the impact of regional culture, socio-economic factors and other factors on user issues, so as to formulate more targeted service strategies.

[0065] like Figure 2As shown, further preferably, judging whether the keyword feature set meets the preset classification conditions includes: determining at least two question topics based on the keyword feature set, and keywords with a front-set ranking in importance for each question topic; for each question data, counting the number of keywords with a front-set ranking belonging to each question topic contained in the question data; if the question data contains at least three keywords with a front-set ranking from questions of the same topic, then judging that the question data is related to the question topic, and thus judging that the classification conditions are met; otherwise, then judging that the classification conditions are not met.

[0066] Further preferably, if the classification conditions are met, the question feature with the highest priority in the keyword feature set is determined, including: determining the priority standards for different question topics based on the distribution of question topics in different sub-areas and the attention trends of questioners in different sub-areas on different question topics; determining the question feature with the highest priority in the keyword feature set based on the priority standards.

[0067] The accuracy and efficiency of data classification have been significantly improved through refined keyword feature set analysis. It first determines at least two question topics based on the keyword feature set, and screens out the top-ranked keywords under each topic. Then, for each question data, the system counts the number of the top N (N is a preset value) keywords belonging to each question topic contained therein. If a question data contains at least three of the top N keywords from the same topic question, it is determined that the question data is closely related to the corresponding question topic, and thus meets the classification conditions. This method not only effectively avoids the ambiguity and uncertainty of data classification, but also ensures the objectivity and reliability of the classification results. Through this technical means, the platform can more accurately grasp the core of user problems, providing a solid foundation for subsequent data analysis and service optimization.

[0068] Further preferably, cause feature items and action feature items corresponding to problem features are extracted from invalid response features based on the BERT-LSTM-CRF model, including: extracting features from invalid response features, wherein BERT is used for text encoding, LSTM is used for sequence modeling, and CRF is used for sequence labeling, so as to identify and extract invalid response features, and further parse out cause feature items and action feature items related to problem features from the invalid response features.

[0069] More specifically, the cause feature items: These feature items are designed to reveal the root causes of invalid responses. They may involve unclear question content, missing information from the questioner, insufficient knowledge or skills of the respondent, or limitations of platform rules and mechanisms. By deeply analyzing the cause feature items, the platform can identify the root cause of the problem and take targeted measures to improve it.

[0070] Correspondingly, action feature items: corresponding to the cause feature items, action feature items refer to the specific actions that need to be taken to solve the problem. These actions may include providing more detailed or accurate answers, guiding the questioner to supplement necessary information, improving the professional knowledge and skills of the respondent, or optimizing the rules and processes of the platform. The extraction of action feature items helps the platform to clarify the direction of solving the problem and formulate a practical action plan.

[0071] Further preferably, after extracting the cause feature items and action feature items corresponding to the problem features from the invalid response features based on the BERT-LSTM-CRF model, it also includes: vectorizing the cause feature items and action feature items based on BETR, clustering the extracted cause feature items and action feature items using the K-Means model, measuring the distance between two points using cosine similarity, and determining the optimal number of topics by judging the sum of the distances from all points in each category to the cluster center point, and obtaining the final cause feature items and action feature items after further manual merging and screening; evaluating the importance of each cause feature item based on the clustering results and preset evaluation criteria, and determining the priority of governance activities corresponding to different problem topics based on the importance.

[0072] This embodiment demonstrates powerful data processing and analysis capabilities by integrating the BERT-LSTM-CRF model and the K-Means clustering algorithm. It first uses the BERT-LSTM-CRF model to accurately extract the cause and action feature items in the invalid response, and then uses BETR to represent the vector, which enhances the computability of the feature. K-Means clustering combined with the cosine similarity metric effectively distinguishes different feature items, and determines the optimal number of topics by optimizing the sum of the distances between the cluster center points, ensuring the accuracy and efficiency of clustering. Manual merging and screening further improve the accuracy of the results. Finally, the importance of each cause feature item is evaluated based on the clustering results and preset standards, which provides a scientific basis for the prioritization of problem topic governance activities, helps the platform to quickly respond to and solve user problems, and improve service quality and user experience.

[0073] According to the above embodiment of the present invention, the above data processing method, when implemented specifically, includes the following steps:

[0074] ①Data acquisition and preprocessing

[0075] First, we obtain the question data of the questioners in different sub-regions based on the geographical space division and the reply data of the respondents from the online question-and-answer platform. These data cover the detailed information of the user's question and the reply content of the respondent. In order to ensure the accuracy and availability of the data, we perform preprocessing operations, including removing special symbols and stop words from the question data and reply data, and performing word segmentation to form the data set to be analyzed. This step lays a solid foundation for subsequent data analysis.

[0076] ②Response data classification and invalid response identification

[0077] The BERT model is used to classify the reply data in the data set to be analyzed. The BERT model is a pre-trained model based on Transformer, which can accurately capture the contextual information in the text, and is therefore very suitable for language representation. Combined with the supervised learning algorithm, the present invention identifies the invalid response characteristics of the responder. Specifically, according to the degree to which the reply data solves the problem characteristics of the questioner, the effectiveness of the reply data is divided into four categories: solved problems, promised problems, problems being processed, and unresolved problems. Among them, responses to unresolved problems are regarded as invalid response features. In order to improve the accuracy of classification, the present invention also uses manual annotation to obtain the initial training samples, compares the annotation results, divides the data, and uses the BERT model for training and prediction, and uses the three indicators of recall rate, precision rate, and F1-score to evaluate the classification results.

[0078] ③ Question data topic clustering and keyword feature set determination

[0079] For the question data corresponding to the invalid response features, the LDA topic model is used for topic clustering. The LDA topic model can reveal the potential topic structure in the text, so it is very suitable for mining hot issues that users are concerned about. The present invention determines the optimal number of question topics in the question data based on the topic perplexity, and merges similar topics through manual inspection to form a keyword feature set. In addition, the distribution of different question features in different sub-regions is analyzed based on the spatial distribution characteristics of the keyword feature set, and the attention trend of questioners in different sub-regions to different question topics is evaluated.

[0080] In order to determine whether the keyword feature set meets the preset classification conditions, the present invention determines the question topic and the keywords with the highest importance ranking based on the keyword feature set. For each question data, the number of the first N keywords belonging to each question topic contained in it is counted (N is a preset value). If the question data contains at least three of the first N keywords from the same topic question, it is determined that the question data is related to the question topic and meets the classification conditions.

[0081] ④ Update of keyword feature set

[0082] If the keyword feature set does not meet the classification conditions, an update operation is performed. Specifically, the topic clustering is re-performed based on the new question data and reply data, and the keyword feature set is updated until it meets the classification conditions. This step ensures the accuracy and timeliness of the analysis results.

[0083] ⑤Problem feature extraction and cause and action feature item identification

[0084] After the keyword feature set meets the classification conditions, the question feature with the highest priority is determined. The present invention determines the priority standards of different question topics according to the distribution of question topics in different sub-areas and the attention trend of questioners to different question topics, and determines the question feature with the highest priority in the keyword feature set based on the priority standards.

[0085] Next, the BERT-LSTM-CRF model is used to extract the cause feature items and action feature items corresponding to the problem features from the invalid response features. The BERT model is used for text encoding, LSTM is used for sequence modeling, and CRF is used for sequence labeling. This combination can accurately identify and extract key information from the invalid response features. Furthermore, the cause feature items and action feature items related to the problem features are parsed from the invalid response features. In order to evaluate the importance of each cause feature item, the present invention uses the K-Means model to cluster the extracted cause feature items and action feature items, and determines the priority of the corresponding different problem theme governance activities based on the clustering results and preset evaluation criteria.

[0086] like Figure 3 As shown, based on the same inventive concept, the present invention also proposes a data processing system for a question-and-answer platform, the system comprising:

[0087] A data acquisition module is used to acquire question data of questioners in different sub-regions divided based on geographical space and reply data of responders from the online question-answering platform and perform preprocessing to form a data set to be analyzed; wherein the question data includes at least two question features;

[0088] The first analysis module is used to classify the response data in the analysis data set using the BERT model, and identify the invalid response characteristics of the respondent in combination with the supervised learning algorithm;

[0089] The second analysis module is used to perform topic clustering on the question data corresponding to the invalid response features using the LDA topic model to obtain a keyword feature set, and determine whether the keyword feature set meets the preset classification conditions;

[0090] Among them, the classification conditions are: based on the keyword feature set, determine at least two question topics and the keywords ranked in the front set positions in importance of each question topic; for each question data, count the number of keywords in the front set positions belonging to each question topic contained in the question data; if the question data contains at least three keywords in the front set positions from the same topic question, then it is determined that the question data is related to the question topic, and it is determined to meet the classification conditions; otherwise, it does not meet the classification conditions.

[0091] A data update module is used to update the keyword feature set until it meets the classification conditions when the classification conditions are not met;

[0092] The data processing module is used to determine the problem feature with the highest priority in the keyword feature set when the classification conditions are met, and to extract the cause feature item and action feature item corresponding to the problem feature from the invalid response feature based on the BERT-LSTM-CRF model.

[0093] It should be noted here that each module in the above-mentioned data processing system corresponds to steps S1 to S5 in implementing the above-mentioned data processing method, and the instances and application scenarios implemented by multiple modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned embodiment 1.

[0094] It is understandable that the above-mentioned data processing system can be built into an online question-and-answer platform. By constructing an efficient data processing system, it is intended to optimize the question-and-answer quality and user experience on the online question-and-answer platform.

[0095] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above data processing methods are implemented.

[0096] In another embodiment of the present invention, a computer program product including instructions is provided, which, when executed on a computer, enables the computer to execute the steps of any data processing method in the above embodiments.

[0097] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated.

[0098] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0099] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0100] In addition, each functional module in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0101] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method for a question-and-answer platform, characterized in that: The method comprises: Obtaining question data of questioners in different sub-regions based on the regional spatial division and reply data of responders from the online question-answering platform and preprocessing them to form a data set to be analyzed; wherein the question data includes at least two question features; The BERT model is used to classify the response data in the data set to be analyzed, and the characteristics of invalid responses of the responding party are identified by combining the supervised learning algorithm; Using the LDA topic model to perform topic clustering on the question data corresponding to the invalid response features to obtain a keyword feature set, and determining whether the keyword feature set meets a preset classification condition; If the classification condition is not met, updating the keyword feature set until it meets the condition; If the classification conditions are met, the problem feature with the highest priority in the keyword feature set is determined, and the cause feature item and action feature item corresponding to the problem feature are extracted from the invalid response feature based on the BERT-LSTM-CRF model.

2. The data processing method of a question-and-answer platform according to claim 1, characterized in that: Specifically, the invalid response feature is as follows: based on the degree to which the response data solves the problem characteristics of the questioner, the validity of the response data is divided into four categories: solved problems, promised problems, problems being solved, and unsolved problems. The response to unsolved problems is the invalid response feature.

3. The data processing method of a question-and-answer platform according to claim 1, characterized in that: The BERT model is used to classify the response data in the data set to be analyzed, and the characteristics of invalid responses of the responding party are identified in combination with the supervised learning algorithm, including: Manual annotation to obtain the initial training samples; Compare the coding results of the two annotators and discuss and confirm the samples with inconsistent annotations; Divide the data into training set, validation set and test set according to the preset ratio; The BERT model is used for training and final prediction, and the classification results are evaluated using three indicators: recall rate, precision rate, and F1-score.

4. The data processing method of a question-and-answer platform according to claim 1, characterized in that: The LDA topic model is used to perform topic clustering on the question data corresponding to the invalid response features to obtain a keyword feature set, including: Determining the optimal number of question topics in the question data based on the topic perplexity; Through manual inspection, similar question topics are merged, and the final number of question topics and the invalid response features corresponding to each question topic are determined to form a keyword feature set.

5. The data processing method of a question-and-answer platform according to claim 1, characterized in that: After the question data corresponding to the invalid response features are clustered by topic using the LDA topic model, the method further includes: Analyzing the distribution of different problem features in different sub-areas based on the spatial distribution characteristics of the keyword feature set; and, According to the amount of data related to different question themes in different sub-regions in the question data, the attention trends of questioners in different sub-regions on different question themes are evaluated.

6. The data processing method of a question-and-answer platform according to claim 5, characterized in that: The determining whether the keyword feature set meets the preset classification condition includes: Determine at least two question topics based on the keyword feature set, and keywords with a predetermined ranking in importance for each question topic; For each piece of question data, counting the number of keywords in the question data that are included in the question data and belong to a preset ranking of each question topic; If the question data contains at least three keywords from the same topic question with a preset ranking, the question data is determined to be related to the question topic and meets the classification condition; otherwise, it does not meet the classification condition.

7. The data processing method of a question-and-answer platform according to claim 6, characterized in that: If the classification condition is met, determining the question feature with the highest priority in the keyword feature set includes: Determine the priority criteria for different question topics based on the distribution of question topics in different sub-regions and the attention trends of questioners in different sub-regions on different question topics; The question feature with the highest priority in the keyword feature set is determined based on the priority standard.

8. The data processing method of a question-and-answer platform according to claim 7, characterized in that: The BERT-LSTM-CRF model is used to extract the cause feature item and the action feature item corresponding to the problem feature from the invalid response feature, including: Features are extracted from the invalid response features, wherein BERT is used for text encoding, LSTM is used for sequence modeling, and CRF is used for sequence labeling, so as to identify and extract the invalid response features, and further parse the cause feature items and action feature items related to the problem features from the invalid response features.

9. The data processing method of a question-and-answer platform according to claim 1, characterized in that: After extracting the cause feature item and the action feature item corresponding to the problem feature from the invalid response feature based on the BERT-LSTM-CRF model, the method further includes: Based on BETR, the cause feature items and action feature items are represented by vectors, the extracted cause feature items and action feature items are clustered using the K-Means model, the distance between two points is measured using cosine similarity, and the optimal number of topics is determined by judging the sum of the distances from all points in each category to the cluster center point. After further manual merging and screening, the final cause feature items and action feature items are obtained; According to the clustering results and preset evaluation criteria, the importance of each cause characteristic item is evaluated, and the priority of the corresponding problem theme governance activities is determined based on the importance.

10. A data processing system for a question-and-answer platform, characterized in that: The system comprises: A data acquisition module is used to acquire question data of questioners in different sub-regions divided based on geographical space and reply data of responders from the online question-answering platform and perform preprocessing to form a data set to be analyzed; wherein the question data includes at least two question features; The first analysis module is used to classify the response data in the data set to be analyzed by using the BERT model, and identify the invalid response characteristics of the responding party in combination with the supervised learning algorithm; The second analysis module is used to perform topic clustering on the question data corresponding to the invalid response features using an LDA topic model to obtain a keyword feature set, and determine whether the keyword feature set meets a preset classification condition; A data updating module, used for updating the keyword feature set until it meets the classification condition when the classification condition is not met; The data processing module is used to determine the problem feature with the highest priority in the keyword feature set when the classification conditions are met, and extract the cause feature item and action feature item corresponding to the problem feature from the invalid response feature based on the BERT-LSTM-CRF model.

Images