Question and answer analysis platform based on big data

Abstract

The invention discloses a question and answer analysis platform based on big data, and relates to the technical field of intelligent question and answer, the platform comprises a data acquisition module, a user portrait construction module, a personalized question and answer analysis module and an intelligent recommendation module; multi-dimensional data are acquired through the data acquisition module, a user portrait model is constructed based on a deep learning framework, dynamic description of user basic attributes, behavior characteristics and interest preferences is realized, individual demand differences can be accurately identified, the information matching degree is improved, historical behavior data are analyzed by using a time sequence prediction algorithm, and the user experience is improved. The method comprises the following steps: pre-judging potential demands of a user in advance, triggering dynamic recommendation in combination with a real-time scene, shortening a demand insight period, realizing real-time linkage of demand prediction-content supply, adjusting an answer tone based on an emotion analysis algorithm, providing scene recommendation through strategies such as collaborative filtering and content label matching, enhancing individuation and immersion of user experience, and improving user experience. And the viscosity of the platform is obviously improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent question-answering technology, and specifically to a question-answering analysis platform based on big data. Background Art

[0002] A question-and-answer analysis platform is a tool that uses natural language processing, machine learning and other technologies to collect, organize, analyze and mine various question-and-answer data. It can automatically identify the user's intention to ask questions, extract relevant information from large amounts of text data, and generate accurate and concise answers. In practical applications, the question-and-answer analysis platform can be widely used in intelligent customer service, knowledge management, information retrieval and other fields to help users quickly obtain the information they need, improve work efficiency and service quality. With the advent of the big data era, the amount of data has exploded. Traditional question-and-answer analysis platforms face many challenges in processing massive data, and the question-and-answer analysis platform based on big data has emerged. It integrates the advantages of big data technology and has brought new changes to information processing and knowledge services.

[0003] According to the patent application number 201911060230.6, an intelligent question-and-answer platform is disclosed, including a question acquisition system, a consultation scenario modeling system, an answer acquisition system and a data cloud storage system. The above scheme can rely on the model to portray different consultation scenarios, but the existing question-and-answer platform still has certain defects when used. Most of the existing question-and-answer platforms adopt a universal information push strategy and lack differentiated analysis of individual characteristics such as user age, region, interest preferences, etc., resulting in low information matching efficiency. It relies on static rules or simple keyword matching, and it is difficult to capture dynamic changes in user needs in real time, and it is impossible to predict potential problems. There is a time difference between information supply and actual user needs. Answer generation lacks emotional dimension adaptation, and recommended content is not combined with real-time scenarios. Users cannot obtain an "immersive" interactive experience, and platform satisfaction and stickiness are insufficient. Therefore, it is of great significance to develop a question-and-answer analysis platform based on big data. Summary of the Invention

[0004] The purpose of the present invention is to make up for the shortcomings of the existing technology and provide a question-answering analysis platform based on big data. It can collect multi-dimensional data through the data acquisition module, build a user portrait model based on the deep learning framework, and realize the dynamic characterization of users' basic attributes, behavioral characteristics, and interest preferences. It can accurately identify individual demand differences and improve information matching. It uses time series prediction algorithms to analyze historical behavioral data, predict users' potential needs in advance, and trigger dynamic recommendations in combination with real-time scenarios. It adjusts the tone of answers based on sentiment analysis algorithms, enhances the personalization and immersion of user experience, and significantly improves platform stickiness.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a question-answering analysis platform based on big data, which includes: a data acquisition module, a user portrait construction module, a personalized question-answering analysis module, and an intelligent recommendation module;

[0006] The data collection module uses distributed crawler technology, log tracking technology, and API interfaces to obtain user historical question and answer records, browsing behavior data, and interest preference data from multiple heterogeneous data sources, and then cleans, deduplicates, and standardizes them before storing them in a distributed database.

[0007] The user portrait construction module builds a user portrait model based on a deep learning framework. It reduces the dimension of multi-dimensional data and integrates features through feature engineering to generate a user feature vector that includes basic attributes, behavioral characteristics, interest preferences, and demand predictions.

[0008] The personalized question-and-answer analysis module performs semantic analysis of user questions based on a pre-trained language model, determines the personalized dimension of the question based on the user's feature vector, generates personalized answers through query generation and sequence-to-sequence models, and adjusts the tone of the answer through a sentiment analysis algorithm.

[0009] The intelligent recommendation module provides collaborative filtering recommendations, content-based recommendations, and real-time dynamic recommendations based on the cosine similarity between user feature vectors and question-and-answer content labels. It also displays recommended content through the front-end interface and receives user feedback data to optimize the recommendation model.

[0010] Furthermore, the user portrait model is Calculate the dynamic update weight of the user feature vector, where w t is the feature weight at time t, α is the forgetting factor, the optimal value of which is determined by cross-validation on historical data, λ is the decay coefficient, which is optimized on the training set by the gradient descent algorithm, Δt is the interval between the data generation time and the current time, and s t is the interaction score at time t, which is calculated by weighting the user's question frequency, answer adoption rate and stay time. Based on the dynamically updated weight, the historical feature vector and the current feature vector are weightedly fused to generate an updated user feature vector.

[0011] Furthermore, the multi-source heterogeneous data sources of the data collection module include the platform front-end user interaction log, the back-end business database and the third-party authorized data source. The historical question and answer records include the question text, the answer content, the interaction timestamp and the star rating. The browsing behavior data is collected through the front-end SDK, including the page URL, the length of stay, the scroll depth and the click heat map data. The interest preference data is extracted through natural language processing technology to extract the semantic vectors of the user's actively set tags, historical search keywords and favorited content. The semantic vector is generated by the word embedding model and mapped to the predefined domain ontology. The semantic vector is domain-adapted using the following formula: in, is the basic word vector, M is the domain transfer matrix, which is generated through adversarial training on labeled data in the domain, and β is the adaptation coefficient whose optimal value is determined by grid search on the validation set.

[0012] Furthermore, the basic attributes of the user portrait construction module are obtained through user registration information and third-party authentication APIs, including age stratification, gender classification, regional code and occupation tag. The behavioral characteristics are calculated through a sliding time window to calculate the user activity index. The information acquisition preference is determined through content form and reading depth analysis. The problem domain tendency is classified by historical questions through a topic model. The interest preference tag system adopts a hierarchical classification structure, including first-level field tags, second-level sub-tags and dynamically generated temporary tags. The user activity index is calculated using the following formula: Among them, a i (t) is the activity score of the i-th behavior at time t, w i is the behavior weight, determined by the hierarchical analysis method, D(t) is the user login interval, γ is the decay rate, calculated by maximum likelihood estimation on historical login data, and θ is the activity threshold, set to the median of the historical average login interval.

[0013] Furthermore, the user portrait construction module uses TensorFlow to build a deep neural network model, performs feature dimensionality reduction through autoencoders, and uses the attention mechanism to weightedly fuse multi-dimensional features. The user feature vector calculates the similarity between users through Euclidean distance and cosine similarity. The demand forecast uses an LSTM time series forecasting model, inputs a behavior sequence with a window length of 30 days, and outputs the potential demand probability distribution in the next 7 days. The model is updated every 24 hours through incremental learning.

[0014] Furthermore, the pre-trained language model of the personalized question-and-answer analysis module is BERT-base-chinese, which is fine-tuned to adapt to vertical field question-and-answer scenarios. The semantic parsing includes entity recognition, intent classification and slot filling. Named entity recognition is achieved through a conditional random field model. The sequence-to-sequence model adopts a Transformer architecture, and injects user feature vectors as conditional information in the decoding stage. The sentiment analysis algorithm is based on BERT's sentence-level sentiment classifier, and locates sentiment keywords through the attention weight matrix. The generation probability of the personalized answer is calculated by the following formula: P(y|x,u)=Softmax(W·(Transformer(x)+α·Attention(u))), where x is the input question, u is the user feature vector, α is the user preference weight, which is optimized through the policy gradient algorithm in reinforcement learning, W is the output layer weight matrix, and Attention(u) is the attention representation of user features.

[0015] Furthermore, the search formula generated by the personalized question and answer analysis module quickly locates relevant documents through the inverted index, and uses the BM25 algorithm to calculate the similarity score between the document and the question. The vector space model uses Sentence-BERT to generate semantic vectors of the question and candidate answers, and calculates the correlation through the vector inner product. The answer candidate set is generated through a multi-stage filtering mechanism. First, the top-100 documents are recalled based on keyword matching, and then the top-10 answers are screened by semantic similarity sorting. The semantic reorganization realizes answer generation through template filling and knowledge graph reasoning.

[0016] Furthermore, the real-time dynamic recommendation of the intelligent recommendation module is triggered by monitoring the user behavior stream data, and when it is detected that the user stays on the same topic page for more than a threshold time, the context-aware recommendation is started.

[0017] Compared with existing technologies, this big data-based question-answering analysis platform has the following beneficial effects:

[0018] The present invention collects multi-dimensional data through the data acquisition module, builds a user portrait model based on the deep learning framework, and realizes the dynamic characterization of users' basic attributes, behavioral characteristics, and interest preferences. It can accurately identify individual demand differences and improve information matching. It uses the time series prediction algorithm to analyze historical behavioral data, predict users' potential needs in advance, and trigger dynamic recommendations in combination with real-time scenarios, shortening the demand insight cycle and realizing the real-time linkage of "demand prediction-content supply". It adjusts the answer tone based on the sentiment analysis algorithm, and provides scenario-based recommendations through collaborative filtering, content tag matching and other strategies, thereby enhancing the personalization and immersion of user experience and significantly improving platform stickiness.

[0019] Other advantages, objects and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art based on an examination of the following or may be learned from the practice of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0021] Figure 1 This is a structural diagram of a question-answering analysis platform based on big data;

[0022] Figure 2 This is a workflow diagram of a question-answering analysis platform based on big data. DETAILED DESCRIPTION

[0023] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0024] Example 1

[0025] To improve the user shopping experience and customer service efficiency, a large e-commerce platform introduced a big data-based question-and-answer analysis platform. The platform needs to process massive amounts of user behavior data and provide personalized question-and-answer services and product recommendations for users with different consumption habits, regional characteristics, and interest preferences. For example, the platform needs to accurately identify users' shopping intentions, respond to user inquiries in real time, and predict users' potential needs to improve user conversion rates and platform stickiness. See Figure 1 and Figure 2 The implementation process of this embodiment is as follows:

[0026] The platform's customer service chat logs are captured through distributed crawler technology, and user question texts, customer service answer content, interaction timestamps, and user reviews are extracted and stored in a distributed database. The front-end SDK tracking technology is used to collect user stay time, scroll depth, and click heat map data on the product details page, and the data is transmitted to the data center in real time through the log parsing engine. The third-party authorized data source is obtained through the API interface, and natural language processing technology is combined to extract user-active tags, historical search keywords, and semantic vectors of the titles of favorite / liked products. The original data is cleaned (removing duplicate records and outliers), deduplicated, and standardized (unified time format and product category coding).

[0027] According to the characteristics of the e-commerce field, the formula The basic word vectors are adapted, where the domain transfer matrix M is generated on e-commerce labeled data (such as product descriptions and user reviews) through adversarial training, and the adaptation coefficient β is optimized on the validation set through grid search, making the vector representation of words such as "price / performance" and "promotion" closer to e-commerce scenarios.

[0028] Through user registration information (such as age, gender, and region) and third-party authentication APIs (such as real-name authentication interfaces), we can obtain age stratification (such as "18-24 years old" and "25-30 years old"), regional codes (such as "North China" and "Yangtze River Delta"), and occupational tags (such as "student" and "newcomer"). We use a sliding time window and use the formula Quantify user activity, where the behavior weight w i The decay rate γ of the login interval D(t) is determined by the hierarchical analysis method (for example, the weight of "consulting customer service" is higher than that of "browsing products"), and the historical login data is fitted by maximum likelihood estimation. The activity threshold θ is set to the median of the historical average login interval.

[0029] Analyze users' preferences for content format (such as graphic details pages, video introductions) and reading depth (such as whether they have fully browsed product parameters), combine topic models to classify historical questions, determine users' problem area tendencies (such as "logistics consulting" and "product functions"), and build a hierarchical label system. The first-level label is "product category", the second-level sub-label is "brand", and dynamic temporary labels are points of interest captured in real time (such as users' recent frequent searches for "sunscreen" generate the "summer skin care" label).

[0030] The user portrait model is based on the formula Dynamically adjust feature weights. For example, if a user frequently browses products related to "laptops" recently, the weight of their interest preference tag "digital products" is s. t (calculated by browsing time and search frequency weighted) is improved, while the weight of historical "clothing" browsing data decays exponentially as Δt increases (α is determined by cross-validation and λ is optimized by gradient descent).

[0031] Using the LSTM time series prediction model, we input a 30-day behavior sequence (such as browsing categories, consulting keywords, and purchase frequency) and output the probability distribution of potential demand in the next 7 days (such as the probability of "buying a laptop" is 65%). The model is updated daily through incremental learning to capture changes in user demand in real time.

[0032] When a user asks "Is this down jacket suitable for wearing in Northeast China in winter?", the personalized question-answering analysis module uses the BERT-base-chinese pre-trained model to perform entity recognition ("down jacket", "Northeast China winter"), intent classification (judged as "suitability consultation") and slot filling (extracting "usage scenario = Northeast China winter", "product type = down jacket"). The conditional random field model optimizes the named entity boundary recognition (such as distinguishing "Northeast China" as a regional entity and "winter" as a time entity), and combines the user feature vector (such as the regional code "Northeast China", high activity index, and historical search for "warm clothing") to determine the personalized dimensions of the question as "regional climate adaptation" and "warmth performance requirements".

[0033] Answer generation: We use the inverted index to locate product detail pages and customer service knowledge base documents containing the phrase "down jackets for Northeast China in winter." We use the BM25 algorithm to calculate similarity. After recalling the top 100 documents, we use Sentence-BERT to generate semantic vectors for the question and candidate answers (e.g., "a filling volume of 250g or more is suitable for temperatures of -20°C"). We then filter the top 10 answers by sorting them using the inner product of the vectors.

[0034] Based on the Transformer architecture, the user feature vector u is injected during the decoding phase, and personalized answers are generated using the formula P(y|x,u)=Softmax(W·(Transformer(x)+α·Attention(u))). The user preference weight α is optimized through reinforcement learning. If the user's historical answer adoption rate is high, the α weight is increased to highlight their interest preferences (for example, "The XX brand down jacket you are interested in contains 90% white duck down and has been tested to withstand temperatures as low as -30°C").

[0035] The BERT-based sentence-level sentiment classifier identifies the emotional tendencies in user questions (such as "Is it suitable?" which carries a cautious tone). It locates keywords such as "suitable" and "Northeast winter" through the attention weight matrix, and adjusts the tone of the answer to "affirmation + detailed explanation" (such as "Absolutely suitable! This down jacket uses three-layer heat-locking technology, which is very suitable for the severe cold weather in Northeast China").

[0036] The intelligent recommendation module combines user feature vectors with real-time behavior to perform recommendations, calculates the cosine similarity between user feature vectors and similar users (such as those in the Northeast region, the same age group, and those with similar browsing history), and recommends answers (such as "snow boot matching suggestions") and products (such as "down jacket cleaner") that are highly adopted by similar users. Based on the matching degree between question and answer content tags (such as "winter wear" and "warm equipment") and user interest tags, it pushes relevant product review videos and promotional information (such as "discount coupons are applicable to down jacket categories").

[0037] Monitor user behavior flow data. When it is detected that the user stays on the "down jacket details page" for more than 5 minutes, context-aware recommendations are triggered. A floating window pops up on the front-end interface to recommend related products (such as "fleece-lined pants" and "cold-proof gloves"). The recommended content is obtained by calculating the cosine similarity between user features and product labels. "Click to view the real evaluation of this down jacket among Northeast users" links to the highly praised answers containing "Northeast user feedback" in historical questions and answers. The user's click, collection, and purchase behavior data on the recommended content are transmitted back to the recommendation model in real time, and the parameters are updated through the gradient descent algorithm to form a "recommendation-feedback-optimization" closed loop.

[0038] The platform realizes user interaction through the front-end interface, and the back-end adopts microservice architecture to deploy various modules. The data collection module is connected with the distributed database in real time, supporting the processing of 100,000 data writes per second. The user portrait construction module deploys the deep neural network model through TensorFlowServing to achieve millisecond-level generation of feature vectors. The personalized question and answer analysis and intelligent recommendation module are elastically expanded through the Kubernetes cluster to ensure that the response delay is less than 500ms in high-concurrency scenarios. The platform supports real-time linkage of "demand forecasting-content supply". After the LSTM model predicts the user's potential needs (such as "Spring Festival gift-giving needs"), the intelligent recommendation module triggers the recommendation of New Year products in advance. At the same time, the personalized question and answer module generates special answers such as "Spring Festival Gift Box Purchasing Guide".

[0039] In summary, this embodiment, through the full-process implementation of patented technology, achieves the accurate capture of user needs in e-commerce scenarios, the intelligent generation of personalized questions and answers, and the efficient linkage of real-time dynamic recommendations. The platform not only solves the pain points of "universal push and static rule matching" of traditional question-and-answer systems, but also builds a closed loop of "data-driven-model optimization-experience upgrade" through the deep integration of big data and deep learning, providing a reusable technical paradigm for the e-commerce industry to improve user experience and business efficiency. In the future, it can be further expanded to vertical fields such as finance and medical care, and through domain ontology adaptation and model fine-tuning, cross-industry intelligent question-and-answer and recommendation applications can be realized.

[0040] Example 2

[0041] To optimize the patient consultation experience and improve physician service efficiency, a healthcare platform has introduced a big data-based question-and-answer analysis platform. This platform processes multi-dimensional medical data, including patient consultation history, electronic health records, symptom self-diagnosis data, and medication browsing history. It provides personalized health Q&A and medical resource recommendations for patients of different ages, medical histories, and regions. For example, the platform needs to accurately identify patients' consultation intent, respond to common disease inquiries in real time, and predict potential patient needs to alleviate the shortage of primary healthcare resources and improve patient satisfaction. (See [1] for more information.) Figure 1 and Figure 2The implementation process of this embodiment is as follows:

[0042] The platform's medical and patient consultation logs are captured through distributed crawler technology, and the patient's question text, doctor's answer content, interaction timestamp and patient evaluation are extracted, and stored in a distributed database that complies with HIPAA standards. The front-end SDK is used to collect the patient's stay time on the symptom self-check page, the order of symptom selection, and the click heat map of the drug details page, which are transmitted to the data middle platform through an encrypted channel. Third-party medical authorization data is obtained through the API interface. Combined with natural language processing technology, the health tags actively filled in by patients, historical search keywords and semantic vectors of collected health articles are extracted, the original data is desensitized, and duplicate records and outliers are cleaned.

[0043] According to the characteristics of the medical field, the formula The basic word vectors are adapted, where the domain transfer matrix M is generated on medical annotated data (such as clinical guidelines and drug instructions) through adversarial training, and the adaptation coefficient β is optimized on the validation set through grid search, so that the vector representation of professional terms such as "atrial fibrillation" and "antibiotic resistance" are closer to medical scenarios.

[0044] Through the patient registration information and electronic health records, we can obtain age stratification (such as "children" and "middle-aged and elderly"), gender, regional code (such as "South China") and occupational label (such as "teacher" and "retiree"), and combine the third-party authentication API to supplement basic attributes such as allergy history and surgical history. Using a sliding time window (such as the last 30 days), we can use the formula Quantify the patient's health management activity, where the behavior weight w i The decay rate θ of the login interval D(t) is determined by the hierarchical analysis method (e.g., the weight of “symptom self-examination” is higher than that of “drug browsing”), and the historical login data is fitted by maximum likelihood estimation. The activity threshold θ is set to the median of the historical average login interval.

[0045] Analyze patients' preferences for content format (such as "graphic and text popular science", "video consultation") and reading depth (such as whether they have fully read the medication instructions), combine topic models (such as LDA) to classify historical questions, determine patients' problem area tendencies (such as "chronic disease management", "pediatric diseases"), and build a hierarchical labeling system in the medical field. The first-level label is "disease category" (such as "respiratory system diseases", "endocrine system diseases"), the second-level sub-label is "specific disease" (such as "pneumonia", "diabetes"), and dynamic temporary labels are real-time captured health concerns (such as patients' recent frequent searches for "cough medicine" generate "respiratory tract infection" labels).

[0046] Using the LSTM time series prediction model, a 30-day health behavior sequence (such as symptom self-examination frequency, drug browsing history, and consultation keywords) is input to output the probability distribution of potential health risks in the next 7 days (such as "upper respiratory tract infection risk 35%"). The model is updated daily through incremental learning and combined with the medical knowledge base to generate preventive recommendations.

[0047] The user portrait model is based on the formula Dynamically adjust feature weights. For example, when a patient frequently consults about "hypertension diet" recently, the weight of their interest preference label "cardiovascular health" is s t (calculated by the weight of the number of consultations and the length of stay) increases, while the weight of the historical "cold care" data decays exponentially as Δt increases (α is determined by cross-validation and λ is optimized by gradient descent).

[0048] When a user asks "Can diabetics eat apples?", the personalized question-answering analysis module uses the BERT-base-chinese pre-training model to perform entity recognition ("diabetics", "apples"), intent classification (judged as "dietary taboo consultation") and slot filling (extracting "disease = diabetes", "food = apples"). The conditional random field model optimizes named entity boundary recognition (such as distinguishing the entity relationship between "diabetes" and "patients"), and combines the patient's feature vector (such as age "55 years old", medical history "diabetes for 5 years", and region "Northern region") to determine the personalized dimensions of the question as "disease type", "food glycemic index", and "regional dietary characteristics".

[0049] Answer generation: Use the inverted index to locate medical knowledge base documents containing "diabetes apple diet" (such as excerpts from the "Guidelines for the Prevention and Treatment of Type 2 Diabetes in China") and doctors' historical answer records. Use the BM25 algorithm to calculate similarity. After recalling the top-100 documents, use Sentence-BERT to generate semantic vectors for the question and candidate answers (such as "Apple has a GI value of 36, which is a low-GI food and can be consumed in moderation by diabetics"). Then use the vector inner product sorting to select the top-10 answers.

[0050] Based on the Transformer architecture, the patient feature vector u is injected during the decoding phase, and a personalized answer is generated using the formula P(y|x,u)=Softmax(W·(Transformer(x)+α·Attention(u))). The patient preference weight α is optimized through reinforcement learning. If the patient has historically adopted a "detailed data type" answer, the α weight is increased to highlight their needs (e.g., "Based on your blood sugar control, it is recommended to consume no more than 200g of apples per day, and it is recommended to consume them between meals").

[0051] The BERT-based sentence-level sentiment classifier identifies the emotional tendencies in user questions (e.g., "Can I eat it?" carries an element of anxiety). It uses the attention weight matrix to locate keywords such as "diabetes" and "apple," and adjusts the tone of the answer to be "professional + reassuring" (e.g., "Don't worry too much! Apples are rich in dietary fiber, and eating them in moderation has little effect on blood sugar, but you need to monitor your blood sugar after a meal").

[0052] The intelligent recommendation module combines the patient's health characteristics and real-time behavior to perform multi-layer recommendations, calculates the cosine similarity between the patient's feature vector and similar patients (such as those with the same medical history, age group, and similar symptoms), and recommends answers that are highly adopted by similar patients (such as "Exercise precautions for diabetic patients") and medical resources (such as "Diabetes specialist clinic appointment link"). Based on the matching degree between the question and answer content tags (such as "Diabetic diet" and "Low GI food") and the patient's health tags, it pushes relevant popular science articles (such as "Ten Misconceptions about Diabetes Diet") and online courses (such as "Diabetes Self-Management Training Camp").

[0053] Monitor patient behavior flow data. When it is detected that the user stays on the "Apple Nutrition Facts Page" for more than 3 minutes, context-aware recommendations are triggered. A floating window pops up on the front-end interface to recommend related content, "View other low-GI fruit recommendations" (such as blueberries and grapefruits). The recommended content is obtained by calculating the cosine similarity between the patient's health characteristics and the content label through a formula, "Click to view specific cases of diabetic patients eating apples", and link to the highly praised answers containing "actual cases" in historical questions and answers (such as "Patient Mr. Zhang eats 150g of apples every day and his blood sugar is under stable control"). The patient's click, collection, and reservation behavior data on the recommended content are transmitted back to the recommendation model in real time, and the parameters are updated through the gradient descent algorithm to form a "medical demand insight-precise content supply" closed loop.

[0054] The data collection module is connected to the medical-grade distributed database in real time, supports encrypted transmission and storage of medical data, and meets the requirements of the Personal Information Protection Law. The user portrait construction module implements federated learning through TensorFlowPrivacy, aggregates medical data from multiple institutions without leaking patient privacy, and improves the model's generalization ability. The personalized question and answer analysis module integrates medical knowledge graphs (such as UMLS) and uses entity linking technology to ensure that the answer content complies with the latest clinical guidelines to avoid misleading advice. The platform supports real-time linkage of "risk prediction-medical intervention". After the LSTM model predicts that the patient's "blood sugar control risk is increased", the intelligent recommendation module automatically triggers the "appointment follow-up reminder". At the same time, the personalized question and answer module generates "recent blood sugar monitoring suggestions" and pushes them to the patient.

[0055] To sum up, this embodiment deeply adapts the patented technology to medical and health scenarios, and builds an integrated intelligent service system of "prevention-consultation-intervention" through multi-dimensional medical data integration, dynamic health portrait construction, precise analysis of medical semantics and real-time medical resource recommendations. The platform not only solves the pain points of "generalized replies and delayed responses" in traditional medical Q&A, but also realizes the paradigm upgrade from "passive consultation" to "active health management" through medical-grade applications of big data and deep learning. In the future, it can be further connected with the hospital information system to open up the "online consultation-offline diagnosis and treatment" closed loop, provide technical support for the implementation of tiered diagnosis and treatment policies, and promote the development of smart medical care towards precision and personalization.

[0056] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as above in terms of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art can, without departing from the scope of the technical solution of the present invention, make some changes or modifications to equivalent embodiments using the technical contents disclosed above. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A question-answering analysis platform based on big data, characterized by: The platform includes: data collection module, user portrait construction module, personalized question and answer analysis module and intelligent recommendation module; The data collection module uses distributed crawler technology, log tracking technology, and API interfaces to obtain user historical question and answer records, browsing behavior data, and interest preference data from multiple heterogeneous data sources, and then cleans, deduplicates, and standardizes them before storing them in a distributed database. The user portrait construction module builds a user portrait model based on a deep learning framework. It performs dimensionality reduction and feature fusion on multi-dimensional data through feature engineering to generate a user feature vector that includes basic attributes, behavioral characteristics, interest preferences, and demand predictions. The personalized question-answering analysis module performs semantic analysis of user questions based on a pre-trained language model, determines the personalized dimension of the question based on the user's feature vector, generates personalized answers through search formula generation and sequence-to-sequence modeling, and adjusts the tone of the answer through a sentiment analysis algorithm. The intelligent recommendation module provides collaborative filtering recommendations, content-based recommendations, and real-time dynamic recommendations based on the cosine similarity between user feature vectors and question-and-answer content labels. It also displays recommended content through the front-end interface and receives user feedback data to optimize the recommendation model.

2. A question-answer analysis platform based on big data according to claim 1, characterized in that: The user portrait model is Calculate the dynamic update weight of the user feature vector, where w t is the feature weight at time t, α is the forgetting factor, the optimal value is determined by cross-validation on historical data, λ is the decay coefficient, Δt is the interval between the data generation time and the current time, s t is the interaction score at time t, which is calculated by weighting the user's question frequency, answer adoption rate and stay time. Based on the dynamically updated weight, the historical feature vector and the current feature vector are weightedly fused to generate an updated user feature vector.

3. A question-answer analysis platform based on big data according to claim 1, characterized in that: The multi-source heterogeneous data sources of the data collection module include the platform front-end user interaction log, the back-end business database and the third-party authorized data source. The historical question and answer records include the question text, answer content, interaction timestamp and star rating. The browsing behavior data is collected through the front-end SDK, including page URL, dwell time, scroll depth and click heat map data. The interest preference data is extracted through natural language processing technology to extract the semantic vectors of user-active tags, historical search keywords and favorited content. The semantic vectors are generated through the word embedding model and mapped to the predefined domain ontology. The semantic vectors are domain-adapted using the following formula: in, is the basic word vector, M is the domain transfer matrix, which is generated through adversarial training on labeled data in the domain, and β is the adaptation coefficient whose optimal value is determined by grid search on the validation set.

4. A question-answer analysis platform based on big data according to claim 1, characterized in that: The basic attributes of the user portrait construction module are obtained through user registration information and third-party authentication APIs, including age stratification, gender classification, regional code, and occupational tags. The behavioral characteristics are calculated through a sliding time window to calculate the user activity index. The information acquisition preference is determined through content form and reading depth analysis. The question domain tendency is classified by historical questions through a topic model. The interest preference tag system adopts a hierarchical classification structure, including first-level field tags, second-level sub-tags, and dynamically generated temporary tags. The user activity index is calculated using the following formula: Among them, a i (t) is the activity score of the i-th behavior at time t, w i is the behavior weight, D(t) is the user login interval, γ is the decay rate, and θ is the activity threshold.

5. The question-answer analysis platform based on big data according to claim 1, characterized in that: The user portrait construction module uses TensorFlow to build a deep neural network model, performs feature dimensionality reduction through an autoencoder, and uses the attention mechanism to weightedly fuse multi-dimensional features. The user feature vector calculates the similarity between users through Euclidean distance and cosine similarity, and the demand forecast adopts the LSTM time series forecast model.

6. A question-answer analysis platform based on big data according to claim 1, characterized in that: The pre-trained language model of the personalized question-and-answer analysis module is BERT-base-chinese, which is fine-tuned to adapt to vertical field question-and-answer scenarios. The semantic parsing includes entity recognition, intent classification and slot filling. Named entity recognition is achieved through a conditional random field model. The sequence-to-sequence model adopts the Transformer architecture, and the user feature vector is injected as conditional information in the decoding stage. The sentiment analysis algorithm is based on BERT's sentence-level sentiment classifier and locates sentiment keywords through the attention weight matrix. The generation probability of the personalized answer is calculated by the following formula: P(y|x,u)=Softmax(W·(Transformer(x)+α·Attention(u))), where x is the input question, u is the user feature vector, α is the user preference weight, W is the output layer weight matrix, and Attention(u) is the attention representation of the user feature.

7. The question-answer analysis platform based on big data according to claim 1, characterized in that: The personalized question-answer analysis module generates search formulas that quickly locate relevant documents through an inverted index and calculates the similarity score between documents and questions using the BM25 algorithm. The vector space model uses Sentence-BERT to generate semantic vectors for questions and candidate answers, and calculates relevance through vector inner products. The candidate answer set is generated through a multi-stage filtering mechanism, and the semantic reorganization achieves answer generation through template filling and knowledge graph reasoning.

8. The question-answer analysis platform based on big data according to claim 1, characterized in that: The real-time dynamic recommendation of the intelligent recommendation module is triggered by monitoring user behavior stream data. When it is detected that the user stays on the same topic page for more than a threshold time, the context-aware recommendation is started.

Images