Intelligent expert recommendation method and system based on CTR estimation and multi-feature fusion

Abstract

The invention discloses an expert intelligent recommendation method and system based on CTR estimation and multi-feature fusion. The method comprises the following steps: acquiring multi-source heterogeneous case data through data acquisition and standardization; case semantic features are extracted through text preprocessing and LDA topic modeling; feature engineering and quantification are carried out on expert basic information, academic backgrounds, historical behaviors and matching degrees with cases; a GBDT + LR fusion model is adopted to realize multi-feature fusion and CTR estimation; generating an expert recommendation list based on CTR scores in combination with rigid filtering and post-processing strategies, and constructing a continuous learning mechanism through user feedback; the system comprises a function module corresponding to the method to realize accurate and fair expert recommendation. The problems that in the prior art, recommendation accuracy is low, the model structure is single, the feature utilization rate is low, and labor-dependent efficiency is low are solved, expert recommendation accuracy, efficiency, fairness and universality are improved, and the method is suitable for college teaching competition, course review, project acceptance and other scenes.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of expert recommendation, and particularly relates to an expert intelligent recommendation method and system based on CTR estimation and multi-feature fusion. BACKGROUND

[0002] With the continuous improvement of education informatization level, the demand for "expert recommendation" in the scenes of university teaching competition, course evaluation, project acceptance, etc. is increasing. Especially in the teaching achievement evaluation and experimental teaching project declaration, the organizers need to quickly select the evaluation experts who best fit the case content from the large number of experts with different professional directions. The traditional allocation method relying on manual experience or rule matching cannot meet the current intelligent and efficient management needs.

[0003] The expert recommendation problem is essentially a typical matching problem, and the core is how to mine the potential association between the case and the expert to improve the accuracy and efficiency of the recommendation. In recent years, technologies such as recommendation system, machine learning, and natural language processing have gradually landed in the education scene, providing new solutions to the expert recommendation problem. For example, click-through rate estimation (CTR Estimation) is widely used in advertising, e-commerce, etc. to predict the user's interest in content, which is essentially a probability classification problem with good explainability and sorting ability. Migrating the CTR modeling idea to the expert recommendation task has high theoretical feasibility and practical value.

[0004] In addition, with the continuous enrichment of teaching case text content, relying only on structured information (such as research direction, unit, and historical participation record) cannot fully depict the real connotation of the case. Therefore, it is necessary to use LDA and other topic modeling methods to mine semantic features from the text to further improve the understanding ability of the recommendation system to "content semantics". Mature methods such as multi-source feature fusion and model structure combination in the recommendation system can also play an important role in the expert recommendation task.

[0005] The current common expert recommendation technology can be roughly divided into three categories: semantic matching method based on topic modeling, label recommendation method based on expert portrait, and rule-based manual assisted recommendation process. Although various schemes have achieved certain application effects in specific scenarios, they all have common problems such as single structure, low automation degree, and inaccurate matching.

[0006] One type of existing method uses topic modeling and expert relationship graph algorithm to realize the recommendation logic ([1]

[0007] Balog K, Azzopardi L, De Rijke M. Formal models for expert finding in enterprise corpora [C] / / Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval. 2006: 43-50.) which mainly includes the following steps: first, LDA topic extraction is performed on the user or case content to obtain the semantic distribution; then, the expert influence map is constructed by combining user interaction behaviors (such as voting, answering, evaluation, etc.); finally, the recommended priority of the expert under a specific topic is calculated through link analysis or centrality sorting. This method is commonly used in question and answer communities, knowledge sharing platforms, etc. It can capture certain semantic preferences and social attributes. However, this method does not introduce the CTR modeling idea, and the recommendation result cannot directly reflect the "matching probability"; at the same time, its modeling ability for structured behavior characteristics is insufficient, and the model training is highly dependent on the specific community relationship graph, making it difficult to migrate to the education review task.

[0008] Another typical method is to construct an expert portrait and match attributes as the core ([2] Tang J, Zhang J, Yao L, et al. Arnetminer: extraction and mining of academic social networks [C] / / Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. 2008: 990-998.), which collects the academic achievements, research directions, project experiences, etc. of experts to form a multi-label or feature vector representation, and then matches it with the basic labels of the teaching cases. The advantage of this method is that the structure is simple, the implementation cost is low, and it is suitable for rapid deployment in teaching platforms. However, it relies on static information, does not perform semantic mining on case texts, and does not introduce machine learning methods for model training. The recommendation strategy is difficult to adaptively optimize according to data feedback, and there are problems such as "label generalization" and "recommended result solidification".

[0009] Some teaching management systems use a semi-automatic recommendation process of rule configuration + administrator review. The system can filter according to basic information such as expert unit, research direction, and past task participation records, and provide a candidate list for manual confirmation by administrators. Although this method is controllable, it is low in efficiency, especially when the number of cases or the size of the expert database is large, the workload is huge, the recommendation accuracy is limited by the experience of administrators, and lacks intelligent support.

[0010] In summary, the prior art has the following disadvantages:

[0011] (1) The accuracy of expert recommendation is low, and the matching is not accurate enough. Traditional systems mostly use keyword or tag matching to associate experts with cases, lack deep semantic understanding, and cannot fully reflect the matching degree of experts and cases in research fields, content styles, and professional backgrounds. Often, the recommended results do not meet the actual needs.

[0012] (2) The model structure is single, and it is difficult to balance the prediction ability and the result explainability. For example, the traditional logistic regression model, although it has strong result explainability, when facing complex and multi-dimensional input features, it shows insufficient modeling ability and difficulty in mining cross-relationships between features. While more complex models such as deep neural networks are often not suitable for educational management and other scenarios that require result review due to their "black box" nature.

[0013] (3) It is difficult to effectively integrate case text semantics and multi-dimensional features such as expert background, and the utilization rate of features is low. Most systems only rely on the basic information or historical labels of experts to build recommendation models, without deep semantic modeling of teaching case texts, resulting in a rough recommendation logic and an inability to truly understand case requirements from a content level.

[0014] (4) The recommendation process relies on manual judgment, which is low in efficiency and high in operation cost. In traditional review systems, expert allocation often relies on manual screening and one-by-one comparison by administrators. Especially when the number of cases is large, the allocation efficiency is low, errors are easy to occur, and a large amount of human resources is consumed, which seriously restricts the operation efficiency of the system. SUMMARY

[0015] In order to overcome the defects existing in the prior art, the purpose of the present application is to provide an expert intelligent recommendation method and system based on CTR estimation and multi-feature fusion, which acquires multi-source heterogeneous case data through data acquisition and standardization; extracts case semantic features through text preprocessing and LDA topic modeling; performs feature engineering and quantization on expert basic information, academic background, historical behavior and matching degree with cases; realizes multi-feature fusion and CTR estimation by using GBDT+LR fusion model; generates an expert recommendation list based on CTR score combined with hard filtering and post-processing strategy, and constructs a continuous learning mechanism through user feedback; effectively improves the accuracy, efficiency, fairness and universality of expert recommendation, and is suitable for scenes such as university teaching competition, course review, project acceptance and the like.

[0016] In order to achieve the above purpose, the present application adopts the following technical means:

[0017] An expert intelligent recommendation method based on CTR estimation, specifically comprising the following steps:

[0018] Step 1. Collecting structured data and unstructured text of teaching cases;

[0019] Step 2. Carrying out word segmentation, stop word filtering and LDA topic modeling on the text obtained in step 1 to extract semantic feature vectors;

[0020] Step 3. Quantizing expert basic attributes, historical review behavior and matching features with cases to obtain expert multi-dimensional features and expert-case matching degree features;

[0021] Step 4. Fusing the case semantic feature vectors obtained in step 2, the expert multi-dimensional features and the expert-case matching degree features obtained in step 3, and inputting a GBDT+LR fusion model to predict CTR score;

[0022] Step 5. Ranking according to the CTR score obtained in step 4 and applying load balancing and diversity strategy to generate Top-N expert recommendation list and recommendation basis;

[0023] Step 6. Continuously optimizing the model based on the feedback data of the user on the Top-N expert recommendation list and the recommendation basis generated in step 5.

[0024] The specific method of step 1 is: receiving case data in the teaching competition, course review or project acceptance scene, the case data including structured information and unstructured text content; using HTTPS encryption communication protocol, realizing data docking with external platform through RESTful API interface based on SFTP or OAuth2.0 authorization; verifying the data type and format of the received case data, establishing an error handling mechanism including multi-layer data verification, exception capture, log recording, automatic retry and alarm notification, and uniformly storing the standardized structured metadata and unstructured text content to the data warehouse.

[0025] The specific method of step 2 is: using Chinese word segmentation tool to segment the unstructured text content obtained in step 1, and combining with the field self-defined dictionary to optimize the segmentation result; using the general stop word table to filter meaningless function words in the segmentation result, removing punctuation, numbers and special symbols in the text through regular expression or specific character set, forming a bag-of-words model; based on the LDA topic model, the bag-of-words model is topic modeled, the optimal number of topics K is determined through cross-validation and grid search, the LDA model parameters are optimized with perplexity and consistency score as evaluation indexes, and the Top-N topics corresponding to the optimal number of topics are selected as the semantic feature vector of the case.

[0026] The specific method of step 3 is: extracting the basic information and academic background characteristics of experts, the basic information including name, unit, affiliated college, professional and technical title, the academic background characteristics including highest education, graduate school, research field and subdivided research direction; quantifying the historical behavior data of experts, including using time decay weighting and project level weighting to count the number of reviews, calculating the review quality score combined with manager evaluation, anonymous feedback of the reviewed party and achievement transformation, and analyzing the preference of experts to case categories through collaborative filtering or matrix decomposition method; calculating the cosine similarity of case semantic feature vector and expert research direction, constructing unit matching, professional matching and title matching features, forming expert-case matching degree features; the time decay weighting uses exponential decay function or linear decay function, in the project level weighting, the national project review weight coefficient > the provincial project review weight coefficient > the school project review weight coefficient.

[0027] The specific method of step 4 is to fuse the case semantic feature vector obtained in step 2, the expert multi-dimensional feature obtained in step 3 and the expert-case matching degree feature, to form a high-dimensional feature vector by using One-Hot coding or Embedding conversion for the category type feature; a GBDT+LR fusion model is constructed, the high-dimensional feature vector is trained by using the GBDT model, the leaf node index of the GBDT model is taken as a discrete feature, and the leaf node index and the original sparse feature vector are input into the LR model; a training set is constructed by using historical “expert-case-whether successful matching” data, the sample imbalance problem is handled by using oversampling or undersampling, the model is trained by using a cross-entropy loss function and a gradient descent optimizer, and the hyperparameters of the model are optimized by using grid search or random search, wherein the hyperparameters of the GBDT model include the number of trees, the learning rate, the maximum depth of the tree and the minimum data amount of the leaf node, and the hyperparameters of the LR model include the regularization strength and the regularization type.

[0028] The specific method of step 5 is to construct a “case-expert” feature vector for a new case and all experts in the expert library, input the GBDT+LR model trained in step 4 to obtain a CTR score; the experts are hard filtered based on avoidance rules, availability rules and qualification rules, and the filtered experts are sorted in descending order according to the CTR scores; a load balancing strategy is introduced to reduce the weight of the CTR score of an expert with high task load and increase the weight of the CTR score of an expert with low task load, the MMR algorithm or a clustering-based algorithm is used to ensure recommendation diversity, and a Top-N expert recommendation list is generated.

[0029] The specific method of step 6 is to display the Top-N expert recommendation list and the recommendation basis, collect user explicit feedback and implicit feedback, the user explicit feedback includes user acceptance / rejection operation on the recommended experts, manual adjustment of the recommendation list and evaluation score of the recommendation reason, and the user implicit feedback includes records of experts completing the review and subsequent achievements of the reviewed cases; the accepted recommendation of the user or the completed review task of the expert is taken as a positive sample, the rejected recommendation of the user or the rejected review task of the expert is taken as a negative sample, and the manual adjustment result of the user is taken as a true label; new samples are added to the training set by using an incremental learning or a rolling window strategy to retrain the model, and a periodic fine-tuning period is set to realize continuous optimization of the model.

[0030] The load balancing strategy of step 5 is to dynamically adjust the CTR weight according to the current task amount of the expert, and the higher the task amount, the greater the weight attenuation coefficient; the CTR score can be dynamically adjusted according to the case urgency, the expert resource scarcity or the historical recommendation success rate.

[0031] An expert intelligent recommendation system based on CTR estimation and multi-feature fusion, comprising:

[0032] The case data collection module receives case data in the teaching competition, course review or project acceptance scene, adopts an HTTPS encryption communication protocol and a RESTful API interface based on SFTP or OAuth2.0 authorization to realize data docking, verifies and standardizes the case data, and uniformly stores the standardized data to a data warehouse;

[0033] The text preprocessing and theme extraction module is used to perform step 2, adopts a Chinese word segmentation tool combined with a domain self-defined dictionary to segment unstructured text, uses a general stop word table and symbol cleaning to form a bag-of-words model, extracts a case semantic feature vector based on an LDA theme model and optimizes model parameters through cross-validation and grid search;

[0034] The expert feature engineering module is used to perform step 3, extracts expert basic information and academic background features, quantifies expert historical review times, review quality scores and preferences, and calculates expert-case matching degree features;

[0035] The feature fusion and modeling module is used to perform step 4, fuses case semantic features, expert multi-dimensional features and matching degree features, encodes and converts category type features, constructs and trains a GBDT+LR fusion model, processes sample imbalance problems and optimizes model hyperparameters;

[0036] The CTR prediction and sorting module is used to perform step 5, constructs a “case-expert” feature vector of a new case and an expert and predicts a CTR score, filters experts through hard filtering rules, sorts experts in combination with a load balancing and diversity strategy, and generates a Top-N expert recommendation list;

[0037] The recommendation result output module is used to perform step 6, displays an expert recommendation list and a recommendation basis, collects user feedback data, converts the feedback data into training samples, and realizes continuous learning and optimization of the model by using an incremental learning or a rolling window strategy and a regular fine-tuning period.

[0038] The case data collection module is also used for TLS / SSL encryption, identity authentication and desensitization processing of sensitive data, and the identity authentication adopts an API Key / Secret or OAuth2.0 authorization mode.

[0039] The text preprocessing and theme extraction module is also used for regular updating of a domain self-defined dictionary, and the updating is based on analysis of high-frequency words and new concept words of newly added case data.

[0040] Compared with the prior art, the present application has the following beneficial effects:

[0041] (1) Innovative introduction of CTR prediction modeling method significantly improves the accuracy and practicality of recommendations. This invention breaks through the traditional expert recommendation method based on keyword matching, rule filtering, or semantic similarity calculation, and for the first time models the "expert recommendation" task as a click-through rate (CTR) prediction problem. This method is essentially transformed into predicting the probability that an expert is "suitable" or "acceptable" for a certain teaching case, effectively introducing historical review behavior data as a training basis, thereby forming a data-driven recommendation decision logic. By modeling from the perspective of CTR, not only is the accuracy of recommendations and the rationality of ranking improved, but the model also has the ability to learn continuously and update adaptively, better addressing the problems of diverse case content and complex expert selection in teaching scenarios.

[0042] (2) A GBDT+LR fusion structure is adopted to balance the predictive power and interpretability of the model. This invention introduces the mature "GBDT+LR" model fusion architecture from the industry. GBDT (Gradient Boosting Decision Tree) is used to capture the nonlinear interaction relationships between multidimensional features and automatically generate complex feature combinations; the LR (Logistic Regression) model predicts the matching probability of expert-case combinations while retaining interpretability. This fusion scheme not only solves the problem of limited feature expression capabilities of traditional logistic regression models, but also avoids the defect of black-box models (such as deep neural networks) in educational scenarios where results cannot be reviewed. Ultimately, a balance between performance and controllability is achieved in the recommendation results, making it easier for system administrators to understand and intervene.

[0043] (3) Deeply integrate textual thematic semantics and structural features to enhance feature expression capabilities. This invention designs a multi-feature fusion mechanism covering teaching case texts, expert attribute information, and review history. The text semantic dimension uses an LDA topic model to model the case content, extracting its distribution across multiple latent topic dimensions as semantic features. Simultaneously, a custom dictionary and stop word filtering mechanism effectively improve the professionalism and accuracy of Chinese corpus processing. Furthermore, the system introduces structured features such as expert research direction, unit matching degree, and historical review behavior to jointly construct a multi-dimensional sample space. Compared to existing solutions that only use keywords or tags, the feature system proposed in this invention is more expressive and can more accurately depict the true matching relationship between experts and cases.

[0044] In summary, this invention innovatively models the expert recommendation problem as a CTR prediction problem and introduces a GBDT+LR ensemble learning model to deeply integrate multi-source heterogeneous features, constructing a closed-loop automated process to reduce manual intervention. At the same time, it introduces load balancing and diversity strategies to achieve reasonable task allocation and multiple review perspectives, thereby reducing operating costs and significantly improving the accuracy, efficiency, and fairness of expert recommendations. Attached Figure Description

[0045] Figure 1 is the overall architecture diagram of the expert intelligent recommendation system of the present application.

[0046] Figure 2 is the comparison diagram of LDA model perplexity and consistency index under different subject number (K value) of the present application.

[0047] Figure 3 is the comparison diagram of LDA model extraction results before and after adding stop words, wherein, Figure 3 (a) is the extraction result before adding stop words, Figure 3 (b) is the extraction result after adding stop words.

[0048] Figure 4 is the LDA model keyword extraction process of the present application, Case1~CaseN are competition cases, and topic1~topicN are the topics extracted by LDA.

[0049] Figure 5 is the structure of the LR+GBDT fusion model of the present application, the input features are split by the GBDT tree to generate interaction features, which are input into the LR together with the original features for classification.

[0050] Figure 6 is the flowchart of the competition case review expert recommendation process of the present application, Author1~Author3 are case authors, and Reviewer1~ReviewerN are review experts. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical scheme and advantages of the present application more clear and obvious, the present application will be further described in detail below in combination with specific embodiments and with reference to the drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present application. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present application.

[0052] Step 1: Collecting structured data and unstructured text of teaching cases to provide high-quality data sources for the system, specifically including:

[0053] Data reception: receiving the case data submitted by the user in the teaching competition, course review and other scenarios, covering structured information such as case name, course category, author unit, achievement abstract, keyword, and unstructured text content such as achievement report;

[0054] Data docking: preferentially using HTTPS encryption communication protocol, seamlessly docking with the university educational administration platform and project management system through RESTful API interface based on SFTP or OAuth2.0 authorization, and supporting JSON / XML standardized data format;

[0055] Data verification and processing: data type and format verification is performed on all fields, and a multi-layer data verification, exception capture, log recording, and automatic retry and alarm notification error handling mechanism is established;

[0056] Data warehousing: unstructured text is used as semantic modeling input, structured metadata is extracted and standardized, and is uniformly stored in the data warehouse. At the same time, sensitive data is encrypted by TLS / SSL, authenticated by API Key / OAuth2.0, and desensitized.

[0057] Step 2: Tokenize the text obtained in step 1, filter stop words, and perform LDA topic modeling to extract semantic feature vectors; This step mines the semantic features of the case text to provide semantic support for subsequent matching, including:

[0058] Chinese word segmentation and vocabulary standardization: use Jieba, HanLP or customized word segmentation service to segment the case text, combine with the domain dictionary (including "teaching achievements", "project declaration" and other education industry terms) to optimize the segmentation results, and update the dictionary regularly based on new case high-frequency words;

[0059] Stop word filtering and symbol cleaning: use the Harbin Institute of Technology stop word list to filter meaningless words such as "de" and "le", remove punctuation, numbers and special symbols through regular expressions, and form a bag-of-words model;

[0060] LDA topic modeling and parameter optimization: extract the distribution vector of the case under the latent topic based on the LDA model, and select the top-N topics as semantic features; optimize the LDA parameters in the following ways:

[0061] Randomly extract part of the data from the case text as the training set;

[0062] Use cross-validation and grid search methods to perform multiple rounds of experiments on the training set;

[0063] Use perplexity (the smaller the better the generalization ability) and consistency score (the higher the better the semantic consistency) as evaluation indicators to determine the optimal number of topics (K=30 in this embodiment).

[0064] Step 3. Quantify the expert's basic attributes, historical review behavior and matching features with the case to obtain the expert's multi-dimensional features and the expert-case matching degree features;

[0065] Basic information and academic background feature extraction: extract basic information such as expert name, unit, affiliated college, professional title, and academic background features such as highest education, graduate school, research field, and sub-research direction;

[0066] Historical behavior data quantification:

[0067] Review times: Statistics of the total number of historical reviews, introduce time decay function (such as exponential decay) to weight recent reviews, and combine project level weighting (national level 1.5, provincial level 1.2, school level 1.0);

[0068] Review quality: The score of the manager's evaluation of the review report quality and timeliness, the anonymous feedback of the evaluated party on the guidance and fairness of the review, and the achievement transformation of the follow-up award and paper publication of the review case, to calculate the review quality score;

[0069] Expert preference: Through collaborative filtering or matrix decomposition method, analyze the "expert-case" historical score matrix, and infer the potential preference of experts for case categories;

[0070] Expert-case matching degree feature construction: Use cosine similarity to calculate the topic similarity between case LDA semantic vector and expert research direction (or historical review case topic distribution), and construct binary / multi-value features of unit matching (whether the same unit), professional matching (professional background matching with case field), and title matching (title association with case level).

[0071] Step 4. Fuse case semantic features, expert features and cross-matching features, input GBDT+LR fusion model to predict CTR score; This step realizes accurate CTR estimation by fusing multi-source features and constructing fusion model, which includes:

[0072] Feature fusion and vectorization: Fuse case LDA semantic vector, expert basic / academic background / historical behavior features and expert-case matching degree features, use One-Hot encoding or Embedding technology to convert professional title, college name and other category features into high-dimensional feature vectors;

[0073] GBDT+LR model construction:

[0074] GBDT layer: Train GBDT model with high-dimensional feature vector, automatically learn feature non-linear relationship and high-order combination feature through serial integration of multiple decision trees, and output leaf node index as discrete feature;

[0075] LR layer: Input the discrete features output by GBDT and the original sparse feature vector into the LR model, use the advantages of LR in handling sparse data and strong interpretability to realize matching probability prediction;

[0076] Model training and parameter tuning:

[0077] Training set construction: Use historical "expert-case-whether successful matching" data as training set, mark "successful matching" as positive sample (label 1) and "unsuccessful matching" as negative sample (label 0), and use oversampling / undersampling to handle sample imbalance;

[0078] Training optimization: Calculate loss with cross-entropy loss function, update parameters iteratively with Adam or SGD gradient descent optimizer;

[0079] Hyperparameter optimization: Use grid search / random search to optimize the number of trees (e.g. 100) of GBDT, learning rate (e.g. 0.1), maximum depth of tree (e.g. 5), and regularization strength (e.g. C=1.0) of LR, and regularization type (e.g. L2).

[0080] Step 5. Sort by CTR score and apply load balancing, diversity strategy to generate recommendation list; this step converts model prediction results into high-quality recommendation list, including:

[0081] CTR estimation: Construct "case-expert" feature vector for new cases and all experts in the expert database, input the trained GBDT+LR model, and output CTR score between 0 and 1 (representing expert-case matching probability);

[0082] Hard filtering: Apply avoidance rules (exclude experts from the same unit as the case author), availability rules (filter busy / experts who have accepted other tasks), and qualification rules (exclude experts who do not meet the requirements of title / professional requirements) to filter experts who meet basic requirements;

[0083] Fine sorting and post-processing:

[0084] Initial sorting: Sort the filtered experts in descending order of CTR score;

[0085] Load balancing: Downweight experts with high current task load and upweight experts with low load to avoid uneven task allocation;

[0086] Recommendation diversity: Use MMR algorithm or clustering-based algorithm to preferentially recommend experts with large differences in unit and research direction from selected experts to ensure diverse review perspectives;

[0087] Top-N list generation: Combine CTR score and post-processing strategies to generate the final Top-N expert recommendation list (e.g. Top-5 / Top-10).

[0088] Step 6. Continuously optimize the model based on user feedback data; this step realizes the display of recommendation results and continuous optimization of the model, including:

[0089] Result display: Display the Top-N expert recommendation list in a visual manner, including expert name, unit, title, CTR score, and recommendation basis (e.g. "high thematic matching degree" "historical review quality is excellent");

[0090] Feedback data collection: Collecting user explicit feedback (accept / reject recommendations, manual adjustment of the list, evaluation score) and implicit feedback (expert completed review records, case follow-up results), recording timestamps, user ID, case ID, expert ID, and other structured information;

[0091] Model continuous learning: Taking user acceptance / expert completed review samples as positive samples, user rejection / expert rejection review samples as negative samples, and manual adjustment results as true labels; using incremental learning (new samples are directly added to the training set) or rolling window strategy (training with the latest data), setting a regular fine-tuning period (such as once a month, adjusted according to data update frequency), to realize model self-adaptive optimization.

[0092] An expert intelligent recommendation system based on CTR estimation and multi-feature fusion, as shown in Figure 1 , comprising:

[0093] Case data collection module, corresponding to method step 1, as the system data entry, used to receive case data, through HTTPS+SFTP / OAuth2.0 RESTful API interface to interface with external platforms, perform data verification, error handling and standardization, store data uniformly to the data warehouse, and encrypt and desensitize sensitive data;

[0094] Text preprocessing and topic extraction module, corresponding to method step 2, used to perform word segmentation (combined with domain-defined dictionary), stop word filtering (Harbin stop word table) and symbol cleaning on unstructured text of the case, extract case semantic feature vector based on LDA model, and optimize LDA parameters (such as determining K=30) through cross-validation and grid search, while regularly updating the domain-defined dictionary;

[0095] Expert feature engineering module, corresponding to method step 3, used to extract expert basic information and academic background features, quantify expert historical review times (time decay + project level weighting), review quality score (multi-dimensional feedback fusion) and preferences, calculate expert-case topic similarity and unit / professional / title matching features, and construct expert feature vector;

[0096] Feature fusion and modeling module, corresponding to method step 4, used to fuse case semantic features, expert multi-dimensional features and matching degree features, perform One-Hot encoding / Embedding conversion on categorical features, construct and train GBDT+LR fusion model, handle sample imbalance problem, and optimize model hyperparameters through grid search / random search;

[0097] CTR prediction and ranking module, corresponding to method step 5, is used to build the "case-expert" feature vector of the new case and the expert, input the model to get the CTR score, filter the experts through the hard filtering rules, combine the load balancing (weight reduction / weighting) and diversity strategy (MMR / clustering) to sort the experts, and generate Top-N recommendation list;

[0098] Recommendation result output module: corresponding to method step 6, is used to display the recommendation list and the recommendation basis, collect user explicit / implicit feedback data, convert the feedback data into training samples, use the incremental learning / rolling window strategy and regular fine-tuning period to realize continuous learning and optimization of the model.

[0099] Embodiment

[0100] I. System architecture

[0101] The system architecture of the present application is shown in Figure 1 The system architecture of the present application is shown in

[0102] II. Specific implementation of each module

[0103] Implementation of case data acquisition module

[0104] In this embodiment, the module is connected to the "teaching competition management platform" of a certain university, and the competition case data is pulled through the RESTful API interface of OAuth2.0 authorization in HTTPS protocol, including the structured metadata (case ID, category, author unit, etc.) and unstructured report text of cases such as "National College Students Electronic Design Competition" and "University Teaching Achievement Evaluation"; The data is verified for format (such as case category needs to be "engineering" "liberal arts" and other preset values), abnormal data is logged and email alarm is triggered; The standardized data is stored in the MySQL data warehouse, and the sensitive information such as author's ID number is desensitized (the first 6 digits + the last 4 digits are retained).

[0105] Implementation of text preprocessing and topic extraction module

[0106] The Jieba word segmentation tool is used for word segmentation of the case report text, and the "education competition custom dictionary" (containing terms such as "experimental teaching", "course design", "innovation achievement", etc.) is loaded; The stop word table of Harbin Institute of Technology is used to filter virtual words, and the regular expression [^a-zA-Z \u4e00-\u9fa5] is used to remove non-Chinese characters, forming a bag-of-words model.

[0107] When constructing the LDA model, 800 of the 1000 historical case texts were randomly selected as the training set, 5-fold cross-validation and grid search were used to test the number of topics K = 10, 20, 30, 40, 50, and the results are shown in Figure 2 As shown in the table, when K = 30, the perplexity is the lowest (1850) and the consistency score is the highest (0.78), so K = 30 is determined; Figure 3 The LDA extraction results before and after adding stop words are compared. After adding, the key terms such as "wireless communication" and "experimental teaching" can be more accurately retained, and noise such as "de" and "le" can be removed.

[0108] Expert feature engineering and quantification module implementation

[0109] 500 expert data were extracted from the expert database, including:

[0110] Basic / academic features: such as "Zhang San, XX University of Electronic Information College, professor, doctor, graduated from XX University of Science and Technology, research direction for wireless communication";

[0111] Historical behavior quantification: the number of reviews is weighted using an exponential decay function w = t / T (t is the time from the current time, T is 365 days), a certain expert reviews 2 national competitions (weight 1.5) and 3 provincial competitions (weight 1.2) in the past year, then the weighted number of reviews = 2x1.5x(365 / 365)+3x1.2x(180 / 365)≈5.1;

[0112] Review quality: a combination of manager's score (85 points), feedback from the reviewed party (90 points), and award rate of reviewed cases (80%), with a weighted score of 85x0.4+90x0.3+80x0.3=85 points;

[0113] Matching degree feature: the cosine similarity between the LDA semantic vector of a "wireless communication" case and the research direction of the expert is 0.85, the unit matching is "different" (feature value 0) and the professional matching is "fit" (feature value 1).

[0114] Feature fusion and model construction module implementation

[0115] Fusion of case LDA semantic vector (30 dimensions), expert features (15 dimensions) and matching degree features (5 dimensions) to form a 50-dimensional feature vector; One-Hot encoding is used for category features such as "title" and "college";

[0116] GBDT+LR model is constructed, with the following parameters: number of trees for GBDT = 100, learning rate = 0.1, maximum depth of tree = 5, minimum data volume of leaf node = 10; regularization type for LR = L2, regularization strength C = 1.0;

[0117] The historical "expert-case-matching result" data (10,000, 3,000 positive samples, and 7,000 negative samples) is used as a training set, the positive samples are expanded to 7,000 by oversampling, and the Adam optimizer and cross-entropy loss function are used for training for 100 rounds;

[0118] The model evaluation results are shown in Table 1. The "LR+GBDT+all features" model (model three) adopted by the application has a Precision of 76.7%, a Recall of 86.2%, an F1-score of 81.2%, and an AUC of 0.89, which is significantly better than the model using only LR (model one) or the model without using LDA features (model two). As shown in Figure 4 and Figure 5 The system first matches the case author keywords, LDA topic keywords, and the expert keyword list to associate a preliminary candidate expert set for each teaching case, and constructs a "case-expert sample pair"

[0119] Implementation of the CTR prediction and expert ranking module

[0120] A new "wireless communication experiment teaching case" enters the system, traverses the 500 experts to construct a "case-expert" feature vector, and inputs the model to obtain the CTR score;

[0121] Application of hard filtering: remove 10 experts from the same unit as the case author, filter 20 experts who are currently busy, exclude 50 experts whose title is lower than "associate professor", and leave 420 experts;

[0122] After sorting in descending order of CTR score, the top 20 experts are post-processed: the expert who has 3 review tasks (CTR=0.92) is weighted to 0.85, and the expert who has only 1 task (CTR=0.88) is weighted to 0.90; 2 experts with different units / research directions from the top 3 experts are selected by MMR algorithm, and the final Top-5 recommendation list is generated as shown in Figure 6 .

[0123] Implementation of the recommendation result output module

[0124] The Top-5 expert list is displayed on the front end of the system, including "expert name: Zhang San, unit: XX University, title: professor, CTR score: 0.90, recommendation basis: theme matching degree 0.85, historical review quality 85 points, current task load low";

[0125] The administrator accepts 3 expert recommendations, rejects 2, and manually adds 1 expert; the system marks the 3 accepted records as positive samples, the 2 rejected records as negative samples, and the 1 manually added record as a positive sample;

[0126] New samples are added to the training set every month, and a rolling window strategy is used to retain nearly 2 years of data, so that the model is fine-tuned once, to ensure the timeliness of the model.

[0127] III. Implementation effect

[0128] In this embodiment, the method and system are applied to the review of a teaching competition of a certain university. Compared with the traditional manual recommendation, (1) the recommendation accuracy is improved by 25%, and the matching degree of the review experts and the cases is improved from 60% to 85%; (2) the recommendation efficiency is improved by 80%, and the single-case expert screening time is shortened from 2 hours to 10 minutes; (3) the balance of expert task allocation is improved by 30%, and no expert has more than 5 review tasks; (4) after 3 months of fine-tuning, the AUC of the model is improved from 0.89 to 0.92, and the self-adaptability is significantly improved.

[0129] Table 1 Model evaluation result table

[0130]

[0131] The above specific embodiments of the present application are only used for illustrative or explanatory purposes of the principles of the present application, and do not constitute a limitation on the present application. Therefore, any modification, equivalent replacement, improvement, etc. made without departing from the spirit and scope of the present application shall be included in the protection scope of the present application. The appended claims of the present application are intended to cover all variations and modifications falling within the scope and boundary of the appended claims or the equivalent forms of such scope and boundary.

Claims

1. A method for expert intelligent recommendation based on CTR estimation, characterized in that, Specifically comprising the following steps: Step 1. Collecting structured data and unstructured text of teaching cases; Step 2. Performing word segmentation, stop word filtering and LDA topic modeling on the text obtained in step 1 to extract semantic feature vectors; Step 3. Quantifying the basic attributes of experts, historical review behaviors and matching features with cases to obtain expert multi-dimensional features and expert-case matching degree features; Step 4. Fusing the case semantic feature vectors obtained in step 2, the expert multi-dimensional features and expert-case matching degree features obtained in step 3, and inputting them into a GBDT+LR fusion model to predict CTR scores; Step 5. Ranking the CTR scores obtained in step 4 and applying load balancing and diversity strategies to generate a Top-N expert recommendation list and a recommendation basis; Step 6. Continuously optimizing the model based on user feedback data on the Top-N expert recommendation list and the recommendation basis generated in step 5. 2.The expert intelligent recommendation method based on CTR estimation of claim 1, wherein, The specific method of step 1 is: receiving case data in the scenarios of teaching competition, course review or project acceptance, the case data including structured information and unstructured text content; using HTTPS encryption communication protocol to realize data docking with external platforms through RESTful API interface based on SFTP or OAuth2.0 authorization; performing data type and format verification on the received case data, establishing error handling mechanism including multi-layer data verification, exception capture, log recording, automatic retry and alarm notification, and uniformly storing the standardized structured metadata and unstructured text content to data warehouse. 3.The expert intelligent recommendation method based on CTR estimation of claim 1, wherein, The specific method of step 2 is: using Chinese word segmentation tool to perform word segmentation on the unstructured text content obtained in step 1, combining with domain-specific dictionary to optimize the word segmentation result; filtering meaningless function words in the word segmentation result using general stop word table, removing punctuation marks, numbers and special symbols in the text through regular expression or specific character set to form bag-of-words model; performing topic modeling on the bag-of-words model based on LDA topic model, determining the optimal number of topics K through cross-validation and grid search, optimizing LDA model parameters with perplexity and consistency score as evaluation indicators, and selecting Top-N topics corresponding to the optimal number of topics as semantic feature vectors of the case. 4.The expert intelligent recommendation method based on CTR estimation of claim 1, wherein, The specific method of step 3 is: extracting basic information and academic background features of experts, the basic information including name, unit, affiliated college, professional and technical title, and the academic background features including highest education, alma mater, research field and subdivided research direction; quantifying historical behavior data of experts, including using time decay weighting and project level weighting to statistically count the number of reviews, calculating review quality scores by combining manager evaluation, anonymous feedback from the reviewed party and achievement transformation, and analyzing the preference of experts for case categories through collaborative filtering or matrix decomposition method. The cosine similarity of the case semantic feature vector and the expert research direction is calculated, unit matching, professional matching, and title matching features are constructed, and an expert-case matching degree feature is formed; the time decay weighting adopts an exponential decay function or a linear decay function, and in the project level weighting, the national-level project review weight coefficient > the provincial-level project review weight coefficient > the school-level project review weight coefficient.

5. The expert intelligent recommendation method based on CTR estimation according to claim 1, characterized in that, The specific method of step 4 is to fuse the case semantic feature vector obtained in step 2, the expert multi-dimensional feature obtained in step 3, and the expert-case matching degree feature, to convert the category type feature by One-Hot coding or Embedding, and form a high-dimensional feature vector; a GBDT+LR fusion model is constructed, the high-dimensional feature vector is trained by using the GBDT model, the leaf node index of the GBDT model is taken as a discrete feature, and the leaf node index and the original sparse feature vector are input into the LR model; a training set is constructed based on historical "expert-case-whether successful matching" data, the sample imbalance problem is handled by oversampling or undersampling, the model is trained by using a cross-entropy loss function and a gradient descent optimizer, and the hyperparameters of the model are optimized by using a grid search or a random search, wherein the hyperparameters of the GBDT model include the number of trees, the learning rate, the maximum depth of the tree, and the minimum data amount of the leaf node, and the hyperparameters of the LR model include the regularization strength and the regularization type. 6.The expert intelligent recommendation method based on CTR estimation of claim 1, wherein, The specific method of step 5 is to construct a "case-expert" feature vector for a new case and all experts in the expert library, input the GBDT+LR model trained in step 4 to obtain a CTR score, perform hard filtering on the experts based on avoidance rules, availability rules, and qualification rules, sort the filtered experts in descending order according to the CTR scores, introduce a load balancing strategy to reduce the CTR score of an expert with high task load and increase the CTR score of an expert with low task load, use an MMR algorithm or a clustering-based algorithm to ensure recommendation diversity, and generate a Top-N expert recommendation list.

7. The expert intelligent recommendation method based on CTR estimation according to claim 1, characterized in that, The specific method of step 6 is to display the Top-N expert recommendation list and the recommendation basis, collect user explicit feedback and implicit feedback, the user explicit feedback includes user acceptance / rejection operation on the recommended experts, manual adjustment of the recommendation list, and evaluation and scoring of the recommendation reasons, and the user implicit feedback includes records of the experts completing the review and subsequent achievements of the reviewed cases; the accepted recommendation of the user or the completed review task of the expert is taken as a positive sample, the rejected recommendation of the user or the rejected review task of the expert is taken as a negative sample, and the manual adjustment result of the user is taken as a true label; new samples are added to the training set by using an incremental learning or a rolling window strategy to retrain the model, and a periodic fine-tuning period is set to realize continuous optimization of the model. 8.The expert intelligent recommendation method based on CTR estimation of claim 6, wherein, The load balancing strategy of step 5 is to dynamically adjust the CTR weight according to the current task amount of the expert, and the higher the task amount, the greater the weight decay coefficient; the CTR score can be dynamically adjusted according to the case urgency, the expert resource scarcity, or the historical recommendation success rate.

9. The recommendation system based on the method of expert intelligent recommendation according to CTR estimation and multi-feature fusion according to any one of claims 1 to 8, characterized in that, The method comprises the steps of: The case data collection module is configured to receive case data in a teaching competition, course review or project acceptance scenario, and to implement data connection by using an HTTPS encryption communication protocol and a RESTful API interface based on SFTP or OAuth2.0 authorization, verify and standardize the case data, and store the standardized data in a data warehouse; The text preprocessing and topic extraction module is configured to perform step 2, to perform word segmentation on unstructured text by using a Chinese word segmentation tool combined with a domain-specific dictionary, to clean the word bag model by using a general stop word table and symbols, to extract a case semantic feature vector based on an LDA topic model, and to optimize model parameters by cross-validation and grid search; The expert feature engineering module is configured to perform step 3, to extract expert basic information and academic background features, to quantify the number of historical reviews, review quality scores and preferences of the experts, and to calculate expert-case matching degree features; The feature fusion and modeling module is configured to perform step 4, to fuse case semantic features, expert multi-dimensional features and matching degree features, to encode and convert category-type features, to build and train a GBDT+LR fusion model, to process sample imbalance problems and optimize model hyperparameters; The CTR prediction and ranking module is configured to perform step 5, to build a "case-expert" feature vector for a new case and an expert and to predict a CTR score, to filter experts by using hard filtering rules, to rank experts by using load balancing and diversity strategies, and to generate a Top-N expert recommendation list; The recommendation result output module is configured to perform step 6, to display the expert recommendation list and the recommendation basis, to collect user feedback data, to convert the feedback data into training samples, and to implement continuous learning and optimization of the model by using an incremental learning or rolling window strategy and a regular fine-tuning period. 10.The expert intelligent recommendation system based on CTR estimation and multi-feature fusion according to claim 9, characterized in that, The case data collection module is further configured to perform TLS / SSL encryption, identity authentication and desensitization processing on sensitive data, and the identity authentication is performed by using an API Key / Secret or OAuth2.0 authorization method; The text preprocessing and topic extraction module is further configured to update the domain-specific dictionary regularly, and the update is based on analysis of high-frequency words and new concept words in newly added case data.

Images