Intelligent complaint prediction method and system based on investment advisor industry, medium and equipment

Abstract

The invention provides an intelligent complaint prediction method and system based on an investment advisor industry, a medium and equipment. The method comprises the following steps: collecting user data and carrying out preprocessing; constructing a feature project for complaint prediction based on the preprocessed user data characteristics; respectively constructing a first prediction sub-model, a second prediction sub-model and a third prediction sub-model based on the feature engineering, and respectively training to obtain corresponding user complaint prediction results; and integrating the user complaint prediction result of each sub-model into a prediction mother model, and obtaining a final complaint prediction result of the user. According to the intelligent complaint prediction method based on the investment advisor industry, through detailed feature extraction and model training, an enterprise can be effectively helped to perform prevention and intervention before customer complaints occur.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular, relates to an intelligent complaint prediction method, system, medium and device based on the investment consulting industry. Background Art

[0002] In the investment advisory industry, customer complaint tendency prediction is an important risk management and service quality improvement tool. At present, a variety of models are applied to the prediction of customer complaint tendency. For example, Logistic regression, as a basic model, is used in many scenarios; deep forest models and LightGBM models are also being tried, among which the LightGBM model performs better in practical applications, and its AUC (area under the curve) reaches 0.769, which is about 3% higher than other models. When predicting customer complaint tendency, multiple complaint-related feature data are selected for modeling, such as user behavior data, transaction information, and incoming channel information. After variable screening, these feature data can more effectively reflect the customer's complaint tendency and improve the accuracy of the prediction.

[0003] However, some existing models may perform well on specific data sets, but their generalization ability may be insufficient when faced with new, unseen data. This means that the model's prediction accuracy may decline in new customer groups or different market environments. Feature engineering is a key step in building effective prediction models, but the current feature selection and processing process is often complicated. If the features are not selected properly or processed accurately, the performance of the model may be affected. Most of the current prediction models are trained based on data from the overall customer group, and the personalized needs and services of individual customers are not considered enough. This may result in the model being unable to accurately predict the complaint tendency of a specific customer in some cases. At the same time, many advanced machine learning and deep learning models are black box models, and it is difficult to intuitively explain their prediction results. This can be a serious problem for the financial industry, as both regulators and customers need to understand the decision-making process and basis of the model.

[0004] Therefore, how to screen out customers with a high tendency to complain through model prediction in the investment advisory industry has become a technical problem that technicians in this field urgently need to solve. Summary of the invention

[0005] In view of the shortcomings of the prior art described above, the purpose of the present invention is to provide an intelligent complaint prediction method, system, medium and device based on the investment consulting industry, which optimizes the general artificial intelligence algorithm through integrated learning technology, combines unstructured data and the objective attributes and subjective expressions of customers, and proposes a new hybrid feature extraction method. In practical applications, good results have been achieved, effectively reducing the actual complaint rate, and improving service efficiency and customer loyalty.

[0006] In a first aspect, the present invention provides an intelligent complaint prediction method based on the investment consulting industry, the method comprising the following steps: collecting user data and preprocessing it;

[0007] Based on the preprocessed user data characteristics, construct feature engineering for complaint prediction;

[0008] Based on the feature engineering, a first prediction sub-model, a second prediction sub-model and a third prediction sub-model are respectively constructed and trained respectively to obtain corresponding user complaint prediction results;

[0009] The user complaint prediction results of each sub-model are integrated into the prediction parent model to obtain the user's final complaint prediction results.

[0010] In an implementation of the first aspect, constructing a first prediction sub-model based on the feature engineering and performing training to obtain a corresponding user complaint prediction result includes the following steps:

[0011] Using a large language model as a pre-training basis for the first prediction sub-model;

[0012] Setting a first business data set and a first business rule;

[0013] Inputting the first business data set into the first prediction sub-model to obtain a first complaint analysis result;

[0014] Training the first prediction sub-model using the first business rule based on the first complaint analysis result;

[0015] Based on the trained first prediction sub-model, a first user complaint prediction result is obtained.

[0016] In an implementation of the first aspect, constructing a second prediction sub-model based on the feature engineering and training it, and obtaining a corresponding user complaint prediction result includes the following steps:

[0017] The BERT model is used as the pre-training basis for the second prediction sub-model;

[0018] Setting a second business data set and a second business rule;

[0019] Inputting the second business data set into the second prediction sub-model to obtain a second complaint analysis result;

[0020] Based on the second complaint analysis result, the second prediction sub-model is trained using the second business rule;

[0021] Based on the trained second prediction sub-model, a second user complaint prediction result is obtained.

[0022] In an implementation of the first aspect, based on the trained second prediction sub-model, after obtaining the second user complaint prediction result, the method further includes the following steps:

[0023] Select a test dataset;

[0024] Inputting the test data set into the second prediction sub-model, and obtaining corresponding test results through different strategies;

[0025] Compare the test prediction results with the actual complaint results to obtain corresponding performance indicators;

[0026] The second prediction sub-model is tuned according to the performance indicator.

[0027] In an implementation of the first aspect, constructing a third prediction sub-model based on the feature engineering and training it, and obtaining a corresponding user complaint prediction result includes the following steps:

[0028] Select the BERT model as the pre-trained model for the third prediction sub-model;

[0029] Set the third business data set;

[0030] Inputting the third business data set into the third prediction sub-model to obtain a third complaint analysis result, wherein the third analysis result includes user direct features and user indirect features;

[0031] fusing the direct features and indirect features of the user data;

[0032] Training the third prediction sub-model based on the fused features;

[0033] Based on the trained third prediction sub-model, a third user complaint prediction result is obtained.

[0034] In an implementation of the first aspect, integrating the user complaint prediction results of each sub-model into a prediction master model, and obtaining the final user complaint prediction results includes the following steps:

[0035] The user complaint prediction results of each sub-model are summarized based on a voting mechanism to obtain the final user complaint prediction result.

[0036] In an implementation of the first aspect, before integrating the user complaint prediction results of each sub-model into the prediction master model, the following steps are also included:

[0037] Assign weights to the user complaint prediction results of each sub-model;

[0038] The weighted average method is used to fuse the prediction results of each sub-model to obtain the fused prediction results.

[0039] In a second aspect, the present invention provides an intelligent complaint prediction system based on the investment consulting industry, the system comprising a collection and processing module, a feature construction module, a model construction module and a complaint prediction module;

[0040] The collection and processing module is used to collect user data and perform preprocessing;

[0041] The feature construction module is used to construct feature engineering for complaint prediction based on the preprocessed user data characteristics;

[0042] The model building module is used to respectively build a first prediction sub-model, a second prediction sub-model and a third prediction sub-model based on the feature engineering and perform training to obtain corresponding user complaint prediction results;

[0043] The complaint prediction module is used to integrate the user complaint prediction results of each sub-model into the prediction parent model to obtain the user's final complaint prediction result.

[0044] In a third aspect, the present invention provides an electronic device, the electronic device comprising: a processor and a memory;

[0045] The memory is used to store computer programs;

[0046] The processor is used to execute the computer program stored in the memory so that the electronic device executes the above-mentioned intelligent complaint prediction method based on the investment advisory industry.

[0047] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by an electronic device, implements the above-mentioned intelligent complaint prediction method based on the investment advisory industry.

[0048] As described above, the intelligent complaint prediction method, system, medium and device based on the investment consulting industry described in the present invention have the following beneficial effects:

[0049] The present invention analyzes the communication text between customers and sales to identify the customer's emotions and tone, thereby determining whether the customer has a tendency to complain. This analysis can help companies take measures before the customer's emotions deteriorate, intervene in advance to deal with potential problems, and thus prevent complaints from happening;

[0050] The present invention extracts key features, such as text features, behavioral features, and speech features, from a large amount of customer interaction data. These features are used to train machine learning models to enable them to identify patterns and trends that may lead to complaints. Through these patterns, companies can better understand the causes of complaints and take corresponding improvement measures;

[0051] The invention can provide personalized service recommendations based on each customer's specific situation and historical interactions. By predicting and reducing complaints, the model helps to improve customer satisfaction and enhance customer loyalty, thereby improving the company's overall service quality and customer relationship management level. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 Shown is a flow chart of an embodiment of the intelligent complaint prediction method based on the investment consulting industry of the present invention;

[0053] Figure 2 Shown is a model architecture diagram of an intelligent complaint prediction method based on the investment consulting industry in one embodiment of the present invention;

[0054] Figure 3a Shown is a first prediction model application diagram of an embodiment of the intelligent complaint prediction method based on the investment consulting industry of the present invention;

[0055] Figure 3b It shows a second prediction model application diagram of an embodiment of the intelligent complaint prediction method based on the investment consulting industry of the present invention;

[0056] Figure 3c It shows a third prediction model application diagram of an intelligent complaint prediction method based on the investment consulting industry in one embodiment of the present invention;

[0057] Figure 3d Shown is a diagram of the parent model verification results of an intelligent complaint prediction method based on the investment consulting industry in one embodiment of the present invention;

[0058] Figure 4 Shown is a schematic diagram of the structure of an intelligent complaint prediction system based on the investment consulting industry in one embodiment of the present invention;

[0059] Figure 5 It is a schematic structural diagram of an electronic device of the present invention in one embodiment. DETAILED DESCRIPTION

[0060] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0061] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0062] The technical solutions in the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0063] like Figure 1 As shown, in one embodiment, the intelligent complaint prediction method based on the investment advisory industry of the present invention includes steps S11 to S14.

[0064] Step S11: Collect user data and perform pre-processing.

[0065] Specifically, the user data includes structured data and unstructured data.

[0066] (1) Structured data

[0067] Structured data refers to data with a fixed format that is easy to search and query, usually stored in a database. In the present invention, it can include the following categories:

[0068] 1) Customer information data:

[0069] Basic personal information: such as name, age, gender, contact information, etc. Financial information: such as account balance, transaction records, asset allocation, etc. Service usage data: such as product holdings, transaction frequency, service satisfaction feedback, etc.

[0070] 2) Transaction data:

[0071] Transaction time, amount, type (buy, sell, transfer, etc.). Transaction status (successful, failed, pending). Counterparty information. Change records of customer accounts, such as capital inflows and outflows. Portfolio performance data, such as yield, volatility, etc.

[0072] (2) Unstructured data

[0073] Unstructured data refers to data that has no fixed format and is not easy to search and query. In the present invention, unstructured data can include the following categories:

[0074] 1) Text data:

[0075] Customer complaint records: including complaint content, complaint time, complaint method, etc. Customer consultation records: such as chat records and email exchanges with customer service, etc. Social media data: such as customer comments and discussions on investment advisory services on social media.

[0076] 2) Voice data:

[0077] Customer service call recording: the call recording between the customer and the customer service, used to extract the content of the complaint or consultation. Voice message: the voice message left by the customer.

[0078] 3) Image data: Scanned copies of documents provided by the customer, such as identity documents, transaction vouchers, etc. Image data of the customer's signature.

[0079] In one embodiment, all chat records between customers and sales on the enterprise WeChat platform, as well as customer information tables and transaction tables, are collected. The above data is preprocessed, specifically including the following steps:

[0080] Step S111, data cleaning: remove invalid or irrelevant data, such as meaningless symbols, expressions, stop words, etc. The purpose is to improve data quality and ensure that subsequent analysis or modeling processes are not affected by noise data.

[0081] Step S112, text formatting: ensure that all text data formats are unified, such as converting to lowercase, removing extra spaces, etc. The purpose is to unify the data format for subsequent processing and analysis.

[0082] Step S113, text length filtering: according to the model input requirements, the text length is limited, and the text exceeding the length is truncated. Purpose: to ensure that the input data meets the model requirements and avoid the difficulty of model processing caused by too long text.

[0083] Step S114: Data labeling: Data labeled by colleagues in the business department. The labeling by colleagues in the business department reflects the business logic and goals, which helps the model learn relevant features. Providing labeled data is conducive to the training and evaluation of supervised learning models.

[0084] Data collection and preprocessing steps are the basis for building efficient and accurate machine learning models. They ensure the quality, consistency, and availability of data, thereby improving the effectiveness of model training and the accuracy of predictions.

[0085] Step S12: construct feature engineering for complaint prediction based on the preprocessed user data characteristics.

[0086] In the intelligent complaint prediction method of the present invention, both structured data and unstructured data need to be properly preprocessed and feature extracted before they can be used by each sub-model. Structured data is usually processed through database query and data analysis tools, while unstructured data needs to be parsed and converted using natural language processing (NLP), speech recognition, image recognition and other technologies. Through these processes, unstructured data can be converted into a form that can be used for machine learning models, and thus used together with structured data for complaint prediction.

[0087] Step S13: Based on the feature engineering, a first prediction sub-model, a second prediction sub-model and a third prediction sub-model are constructed and trained to obtain corresponding user complaint prediction results. Figure 3a to Figure 3c As shown:

[0088] (1) The first prediction sub-model

[0089] Constructing a first prediction sub-model based on the feature engineering and training it, and obtaining the corresponding user complaint prediction results include the following steps:

[0090] Using the Qianwen model as the pre-training basis of the first prediction sub-model;

[0091] Setting a first business data set and a first business rule;

[0092] Inputting the first business data set into the first prediction sub-model to obtain a first complaint analysis result;

[0093] Training the first prediction sub-model using the first business rule based on the first complaint analysis result;

[0094] Based on the trained first prediction sub-model, a first user complaint prediction result is obtained.

[0095] The task of the first prediction sub-model is to parse the input customer information, transaction information and customer speech to identify the customer's emotions, complaint tendency and refund tendency. In one embodiment, the construction and application process of the first prediction sub-model is as follows:

[0096] Step S131: data preparation.

[0097] Organize by customer ID and date. This might be to aggregate data from different chats, emails, or phone calls, based on each customer's daily activity. If a customer has multiple communications on the same day, those communications will be combined into a single paragraph of text, the single-day prompt.

[0098] The purpose of this process is to better understand the overall sentiment and inclination of customers on a specific day by combining the daily speech content of each person into a single daily prompt. This processing helps capture the fluctuations in customer sentiment over time and provide more coherent input data for the model.

[0099] The format of the data input needs to be unified so that the model can parse it correctly. Each input contains the following information: Customer ID: a number that uniquely identifies the customer. Customer information: may include gender, age, region, etc. Transaction information: the products purchased by the customer and the price. Incoming channel information: which platform or channel the customer communicated through. Date: the date the communication took place. Full speech content of the day: all speeches within a day are merged and separated by a specific delimiter (such as "|||").

[0100] The purpose of this process is to ensure that each piece of data contains comprehensive information about the customer so that the model can combine multiple sources of information to make predictions. This multi-dimensional input helps improve the model's predictive power because it takes into account the customer's personal background, transaction behavior, and communication content.

[0101] Step S132: pre-training of large model.

[0102] The Qianwen model is used as the pre-training basis for the first prediction sub-model.

[0103] The first business data set and the first business rule are used to perform prediction output on the first prediction sub-model.

[0104] Input the "single-day prompt" into the Qianwen model to obtain the first complaint analysis result:

[0105] 1) Input content structure

[0106] Customer information: Gender: Male; Region: Liaoning; Age: 35 years old; Transaction information: Purchased product: 1.0 product; Purchase price: 998 yuan; Incoming channel: Kuaishou; Customer comments: Merge the daily comments and use “|||” as a separator.

[0107] 2) Output content

[0108] Return format: JSON format;

[0109] Included fields: Customer sentiment: the value range is "positive / neutral / negative"; Customer complaint tendency: the value range is "high / medium / low"; Customer refund tendency: the value range is "high / medium / low".

[0110] Output format: Output the first complaint analysis result. The model returns the result in JSON format for each prompt, such as {"Customer sentiment": "Neutral", "Customer complaint tendency": "Low", "Customer refund tendency": "Low"}.

[0111] Step S133: large model prediction output.

[0112] 1) Randomly select 20,000 customers who completed transactions in 2023, of which 10,000 have complaint records and 10,000 have no complaint records. For customers who complain, use the chat records from the transaction to the complaint; for customers who have not complained, use the chat records to date.

[0113] 2) First business rule setting: Set rules based on the sentiment and tendency output by the model to predict whether a customer will complain. Use the exhaustive method to try different combinations of thresholds (a, b, c values ​​range from 1 to 10) and logical operators ("or" and "and") to find the best prediction rule. The details are as follows:

[0114] The number of high complaint tendency (denoted as A); the number of high refund tendency (denoted as B); the number of negative emotions (denoted as C).

[0115] When "number of high complaint tendency >= 1" or "number of high refund tendency >= 1" or "number of negative emotions >= 1", it is predicted that the customer will complain.

[0116] Since experts may not be able to directly give specific quantitative indicators (i.e., the specific values ​​of a, b, c), we use an exhaustive method to try all possible combinations. Here, a, b, and c represent the thresholds of high complaint tendency, high refund tendency, and negative sentiment, respectively, and their values ​​range from 1 to 10.

[0117] The present invention covers the following six basic logic combinations, each with different prediction logic:

[0118] 1) A or B or C

[0119] 2) A and B and C

[0120] 3) A or (B and C)

[0121] 4)(A or B) and C

[0122] 5) (A and B) or C

[0123] 6) A and (B or C)

[0124] For each logical combination, we combine all possible values ​​of a, b, and c (1 to 10), resulting in a total of 180 different situations (10 possible values ​​of a × 10 possible values ​​of b × 10 possible values ​​of c).

[0125] For each case, we calculate the prediction accuracy using the following formula:

[0126]

[0127] The prediction accuracy of all situations is sorted, and the indicator corresponding to the situation with the highest accuracy (i.e., the specific values ​​of a, b, and c) will be selected as the comprehensive accuracy indicator.

[0128] The model constructed by using the logical combination and threshold corresponding to the comprehensive accuracy index as the prediction rule is the final first prediction sub-model. The trained first prediction sub-model will be used in actual business scenarios to obtain the first user complaint prediction results and predict the possibility of customer complaints by analyzing the customer communication content.

[0129] (2) The second prediction sub-model.

[0130] Building a second prediction sub-model based on the feature engineering and training it, and obtaining the corresponding user complaint prediction results include the following steps:

[0131] The BERT model is used as the pre-training basis for the second prediction sub-model;

[0132] Setting a second business data set and a second business rule;

[0133] Inputting the second business data set into the second prediction sub-model to obtain a second complaint analysis result;

[0134] Based on the second complaint analysis result, the second prediction sub-model is trained using the second business rule;

[0135] Based on the trained second prediction sub-model, a second user complaint prediction result is obtained.

[0136] Furthermore, based on the trained second prediction sub-model, after obtaining the second user complaint prediction result, the following steps are also included:

[0137] Select a test dataset;

[0138] Inputting the test data set into the second prediction sub-model, and obtaining corresponding test results through different strategies;

[0139] Compare the test prediction results with the actual complaint results to obtain corresponding performance indicators;

[0140] The second prediction sub-model is tuned according to the performance indicator.

[0141] Specifically, the construction and application process of the second prediction sub-model is as follows:

[0142] S231. Model input preparation.

[0143] Each customer's speech is classified into labels including customer sentiment (positive, neutral, negative), complaint tendency (high, medium, low) and refund tendency (high, medium, low). Similarly, sales speeches are also classified into sales sentiment (positive, neutral, negative) and comfort level (high, medium, low).

[0144] S232. Model selection and training.

[0145] The BERT model and the following Chinese pre-training corpus are used as the pre-training basis for the second prediction sub-model.

[0146] roberta-base-finetuned-dianping-chinese

[0147] roberta-base-finetuned-jd-full-chinese

[0148] roberta-base-finetuned-jd-binary-chinese

[0149] chinese-roberta-wwm-ext

[0150] finance-sentiment-zh-base

[0151] The above corpus was pre-trained on different Chinese corpora and has a high correlation with finance and customer speech.

[0152] By adjusting hyperparameters and using different pre-trained models, we can find the model with the highest text classification accuracy and the corresponding hyperparameters as the pre-training basis for the final second prediction sub-model.

[0153] S233, model output.

[0154] The user data and the second business rules are input into the second prediction sub-model to obtain a second complaint analysis result.

[0155] Through the fine-tuned BERT text classification model, the chat data from the previous seven days is used to make five-category predictions every day, and customers are marked as having a high or low tendency to complain based on the second business rule. The second business rule combines customer sentiment, complaint tendency, sales sentiment, and comfort level.

[0156] Output: The output is a table containing customer ID and complaint label, updated daily.

[0157] In one embodiment, the number of days when customer sentiment is negative ≥ a and the number of days when complaint tendency is high ≥ b and the number of days when sales sentiment is negative ≥ c and the number of days when appeasement level is high ≤ d are selected, where the specific values ​​of a, b, c, and d need to be determined based on the test results. If the rule is triggered, the customer is marked as a customer with high complaint tendency; otherwise, it is marked as a customer with low complaint tendency. The output result is a table containing two columns: cust_id (customer ID) and complaint label (1 for high complaint tendency, 0 for low complaint tendency). This table will be updated daily and predictions will be made based on the chat data of the previous seven days.

[0158] S234. Model evaluation and validation.

[0159] In one embodiment, all customers who completed transactions in 2024 are selected, and daily predictions are made based on chat records from January 1, 2024 to July 1, 2024. The prediction results are compared with the actual complaints, and the recall rate, precision rate and F1 score are calculated. The test strategy is as follows:

[0160] Strategy 1 (number of days): Based on a specific threshold, the number of days with negative sentiment, high complaint tendency, and high refund tendency is calculated to mark high complaint tendency customers. The threshold range is 1 to 6.

[0161] Strategy 2 (Number): Based on a specific threshold, the number of negative sentiments, high complaint tendency, and high refund tendency is calculated to mark customers with high complaint tendency. The threshold range is 1 to 10.

[0162] Strategy 3 (weight): Use the time decay weighting method to select customers whose weighted times of customer sentiment, complaint tendency, and refund tendency meet a specific threshold (1 to 10) and mark them as having a high complaint tendency. The time decay coefficient alpha is 0.9, and the weighting coefficient is e^(-alpha*t), where t is 0 for the first day before, 1 for the second day before, and so on.

[0163] There are three types of verification indicators:

[0164] Precision: Measures the proportion of customers predicted to have a high tendency to complain who actually complain.

[0165] Recall: Measures the proportion of customers who are predicted to have a high tendency to complain among those who actually complain.

[0166] F1_score: It is the harmonic mean of precision and recall, which is used to measure the balanced performance of the model. The higher the F1 score, the better the performance of the model.

[0167] Based on the above strategy, the prediction accuracy is verified based on the actual complaint records in the prediction results on January 1, 2024 (the customers screened are those who have spoken and have transaction records on that day), and in the following 1, 3, 7, 14, 21, and 28 days. The verification results are as follows:

[0168] Table 1 Verification results of the second prediction sub-model based on strategy 1

[0169]

[0170]

[0171] Table 2 Verification results of the second prediction sub-model based on strategy 2

[0172]

[0173]

[0174]

[0175] Table 3 Verification results of the second prediction sub-model based on strategy 3

[0176]

[0177]

[0178]

[0179] Similarly, the prediction results and corresponding indicators for each day from January to June 2024 are calculated. Finally, all the results are combined to calculate the average precision, recall and f1 coefficient of each case, and then the three are averaged to obtain the comprehensive index, and the corresponding strategy and threshold with the highest comprehensive index are selected as the final second prediction sub-model.

[0180] (3) The third prediction sub-model

[0181] Constructing a third prediction sub-model based on the feature engineering and training it, and obtaining the corresponding user complaint prediction results include the following steps:

[0182] Select the BERT model as the pre-trained model for the third prediction sub-model;

[0183] Set the third business data set;

[0184] Inputting the third business data set into the third prediction sub-model to obtain a third complaint analysis result, wherein the third analysis result includes user direct features and user indirect features;

[0185] fusing the direct features and indirect features of the user data;

[0186] Training the third prediction sub-model based on the fused features;

[0187] Based on the trained third prediction sub-model, a third user complaint prediction result is obtained.

[0188] Specifically, the construction and application process of the third prediction sub-model is as follows:

[0189] 1) Third business data set processing.

[0190] The process involves creating and extracting features related to each customer to predict the likelihood of their complaint. Feature engineering for the third prediction sub-model is a complex process involving multiple steps to extract and combine different types of features for use by the machine learning model. Relevant features are extracted and classified and labeled based on user data:

[0191] a. Extraction of chat related features

[0192] Customer sentiment value: Statistics include the total number, average, maximum, minimum, median, quantiles (such as 25%, 50%, 75%) and proportion of each emotional state of customers' sentiment classification (positive, neutral, negative) in the previous seven days.

[0193] Complaint tendency value: Similarly, statistics are collected on the complaint tendency (high, medium, and low) in the previous seven days, including the total number, average, maximum, minimum, median, quantiles, and proportion.

[0194] Refund tendency value: The refund tendency is processed in the same statistical way as above.

[0195] Sales sentiment value: Statistics on the salesperson’s emotional performance during the chat, using the same statistical indicators.

[0196] Soothing degree value: evaluates the effectiveness of sales staff in soothing customers during chats, and statistics are also collected.

[0197] b. Extraction of other customer characteristics

[0198] Transaction characteristics: including the customer’s actual average payment price, maximum price, total amount and number of payments.

[0199] Customer information characteristics: including the customer’s asset status, questionnaire score, real age and gender.

[0200] Historical behavior characteristics: record the number of customer complaints and refunds.

[0201] Risk characteristics: such as whether the customer borrows money to invest in stocks.

[0202] Behavioral characteristics: such as the customer’s activity level on the app.

[0203] c. Semantic vector feature aggregation

[0204] Use the BERT model to convert the customer’s speech from the previous day into sentence vectors.

[0205] The average, maximum, minimum, standard deviation, sum, and mode of the sentence vector are calculated to obtain 768*6=4608 direct features.

[0206] The emotion classification is encoded into a 3-dimensional vector and concatenated with the 768-dimensional sentence vector to obtain a 771-dimensional fusion vector.

[0207] Calculate the statistical features of the fusion vector, such as the L2 norm, L1 norm, singular value after singular value decomposition, and the first principal component.

[0208] Direct features: are features extracted directly from customer text data. The customer's text is converted into sentence vectors using the BERT model. Each sentence vector is a 768-dimensional vector that represents the semantic content of the sentence. The following are the statistical operations performed on each sentence vector:

[0209] Mean: The average value of all sentence vector dimensions.

[0210] Maximum value (Max): The maximum value of all sentence vector dimensions.

[0211] Minimum value (Min): The minimum value of all sentence vector dimensions.

[0212] Standard Deviation: The standard deviation of all sentence vector dimensions.

[0213] Sum: The sum of all sentence vector dimensions.

[0214] Mode: The mode of all sentence vectors in their dimension (although in practice vectors do not have a mode, this may refer to the most common value or the value with the highest frequency).

[0215] In this way, each customer's speech the day before will generate 768 direct features, each feature is composed of 6 statistics, and a total of 768*6=4608 direct features are obtained.

[0216] Indirect features: generated by other methods, such as sentiment classification. Sentiment classification is encoded as a 3D vector representing positive, negative, and neutral emotions. The sentiment classification vector is horizontally concatenated with the 768-dimensional semantic vector generated by BERT to form a 771-dimensional fusion vector.

[0217] Statistical features: Calculate various statistical data of these fused vectors, such as the mean, maximum, minimum, standard deviation, sum, and mode of the Euclidean norm (L2 norm), the mean, maximum, minimum, standard deviation, sum, and mode of the Manhattan distance (L1 norm), the mean, maximum, minimum, standard deviation, sum, and mode of the singular values ​​after singular value decomposition, and the first principal component.

[0218] Among them, the calculation formula of the Euclidean norm (L2 norm) is as follows:

[0219]

[0220] The calculation formula of the mean value (mu_L2) is as follows:

[0221]

[0222] The calculation formula of Manhattan distance (L1 norm) is as follows:

[0223]

[0224] The calculation formula for the mean value (mu_L1) is as follows:

[0225]

[0226] The calculation formula for the mean value (mu_SVD) of the singular values ​​after singular value decomposition (SVD) is:

[0227]

[0228] Assume there are n sentence vectors, and the i-th element of each vector is represented by x ij (j=1,…n); i represents the i-th element of the vector; s j Represents the singular value after singular value decomposition of the jth fusion vector; the calculation of the first principal component involves principal component analysis (PCA), and the formula is: find the eigenvalues ​​and eigenvectors of the covariance matrix, and then select the eigenvector corresponding to the largest eigenvalue as the first principal component.

[0229] 2) Model training.

[0230] Select the following machine learning algorithms to build the model:

[0231] Logistic regression: Applicable to binary classification problems and can output probability values.

[0232] Random forest regression prediction: an integrated learning method that can handle nonlinear relationships.

[0233] Multilayer Perceptron (MLP) regression prediction: a feed-forward artificial neural network suitable for recognizing complex patterns.

[0234] XGBoost regression prediction: An integrated learning method based on gradient boosting with high prediction accuracy.

[0235] The goal of the third prediction sub-model training is to build a model that can predict the probability of a customer making a complaint. This probability is a value between 0 and 1, where 1 means that the customer will definitely complain and 0 means that the customer will definitely not complain.

[0236] Each individual model (logistic regression, random forest, multi-layer perceptron, XGBoost) provides a complaint probability value for each customer. For example, suppose we have four models, and the complaint probability values ​​they provide for a certain customer are a, b, c, and d. In order to combine the predictions of these models, we use the weighted average method to calculate the final predicted probability. The weighted average is calculated as follows:

[0237] First, calculate the weight of each model probability value. For each model x, its weight calculation formula is:

[0238]

[0239] Among them, the weight x Represents the probability value of model x.

[0240] Then, calculate the weighted average probability value, the formula is:

[0241] Final Probability x =Weight a ×a+weight b ×b+weight c ×c+weight d ×d

[0242] The final probability value after weighted average will be used to determine whether the customer will complain. If the final probability value is greater than 0.5, it is predicted that the customer will complain (output 1), otherwise it is predicted that the customer will not complain (output 0)

[0243] Furthermore, hyperparameters are model parameters that are not learned from the data but need to be set before training. The value of hyperparameters has a great impact on the performance of the model.

[0244] The joint grid search method is a hyperparameter optimization method that systematically iterates over multiple hyperparameter combinations to find the best parameter settings. In this example, we selected different parameter groups (e.g., parameter group a and parameter group b) for Model 3 and Model 4 to search.

[0245] The R-squared coefficient is a statistical indicator that measures the goodness of fit of a model. The closer its value is to 1, the more variance the model explains. During the hyperparameter adjustment process, we use the R-squared coefficient on the test set as the evaluation criterion.

[0246] By continuously adjusting the hyperparameters, we will find the parameter combination that gives the highest R-squared coefficient, or until the R-squared coefficient exceeds 0.9. This parameter combination will be considered as the final third prediction sub-model.

[0247] In this way, we can ensure that the third prediction sub-model has the best generalization ability and performance in predicting the probability of customer complaints. This process is iterative and may require multiple runs to find the best hyperparameter combination.

[0248] Step S14: Integrate the user complaint prediction results of each sub-model into the prediction parent model to obtain the final user complaint prediction results.

[0249] The prediction results of the three prediction sub-models are combined and a voting mechanism is used to determine the final prediction. If the majority of models predict that a customer will complain, the customer is marked as having a high tendency to complain.

[0250] In the prediction phase, each sub-model predicts the test set and outputs its own prediction results. The parent model combines the prediction results of all sub-models according to predetermined rules to form the final prediction output. Some sub-models may perform better than other models on specific tasks, so these models can be given higher weights. Weights are assigned based on the accuracy of each sub-model on the validation set. Models that perform better are given higher weights. That is, through an iterative process, weights are adjusted based on the performance of the sub-models on the validation set. After adjusting the weights, the performance of the entire integrated model needs to be evaluated on an independent test set. Weights can also be adjusted according to empirical rules.

[0251] The parent model will set a threshold based on performance indicators (such as maximizing the F1 score, balancing the novelty rate and precision rate based on the specific application scenario) to decide when to output the final prediction result. For example, only when all child models predict the positive class will it be finally judged as the positive class.

[0252] In one embodiment, under normal conditions, when the capabilities of the three sub-models are balanced, if the number of models predicted to be high is greater than or equal to 2, then the final output is a high complaint tendency, otherwise it is a low complaint tendency. In special cases, adjustments can be made according to actual conditions:

[0253] 1) When the weight of one of the sub-models is biased, for example, when the business focuses more on time series data, the final output is high only when the number of models with high predictions is greater than or equal to 2 and the prediction result of sub-model 1 is high, otherwise it is low.

[0254] 2) When the weights of two sub-models are biased, for example, when the business focuses more on time series data and text classification, the final output is high only when the prediction results of sub-model 1 and sub-model 2 are both high, otherwise it is low.

[0255] Furthermore, the model verifies its accuracy by comparing the actual complaint rates after 7, 14, and 31 days in the prediction list for each day. The average prediction accuracy rates are 26%, 40%, and 60%, respectively, indicating that the model has good prediction capabilities.

[0256] Each sub-model of the present invention may focus on different aspects of the data. For example, the first prediction sub-model may focus on time series data, the second prediction sub-model may focus on text classification, and the third prediction sub-model may focus on statistical features and semantic analysis. Through this division of labor, each sub-model can capture information that other sub-models may ignore.

[0257] Sub-models are usually trained independently, which means that their training process and parameter optimization do not directly affect other sub-models. This independence can reduce the risk of overfitting between models because each model is trying to understand and predict the data in a different way. Using different algorithms and techniques to build sub-models can increase the diversity of prediction results, which helps improve the robustness of the overall model.

[0258] Although the sub-models are trained independently, their performance in the parent model is interdependent. The strong predictive ability of one sub-model can make up for the weakness of another sub-model. The success of the parent model depends largely on the quality of the sub-models and the complementarity between them. In short, the relationship between the sub-models is complementary and independent, while their relationship with the parent model is integrated and interdependent. With this structure, the parent model is able to leverage the different perspectives of the sub-models to make more accurate predictions.

[0259] The protection scope of the intelligent complaint prediction method based on the investment advisory industry described in the embodiment of the present invention is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present invention are included in the protection scope of the present invention.

[0260] An embodiment of the present invention also provides an intelligent complaint prediction system based on the investment advisory industry, which can implement the intelligent complaint prediction method based on the investment advisory industry described in the present invention. However, the implementation device of the intelligent complaint prediction system based on the investment advisory industry described in the present invention includes but is not limited to the structure of the intelligent complaint prediction system based on the investment advisory industry listed in this embodiment. All structural deformations and replacements of the prior art made according to the principles of the present invention are included in the protection scope of the present invention.

[0261] like Figure 4 As shown, in one embodiment, the intelligent complaint prediction system based on the investment consulting industry of the present invention includes a collection and processing module 41, a feature construction module 42, a model construction module 43 and a complaint prediction module 44;

[0262] The collection and processing module 41 is used to collect user data and perform pre-processing;

[0263] The feature construction module 42 is connected to the acquisition and processing module 41 and is used to construct feature engineering for complaint prediction based on the pre-processed user data characteristics;

[0264] The model construction module 43 is connected to the feature construction model 42, and is used to respectively construct a first prediction sub-model, a second prediction sub-model and a third prediction sub-model based on the feature engineering and perform training to obtain corresponding user complaint prediction results;

[0265] The complaint prediction module 44 is connected to the model building 43 and is used to integrate the user complaint prediction results of each sub-model into the prediction parent model to obtain the user's final complaint prediction result.

[0266] In the several embodiments provided by the present invention, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules or units, which can be electrical, mechanical or other forms.

[0267] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention. For example, the functional modules / units in the various embodiments of the present invention may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0268] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0269] The embodiment of the present invention also provides a computer-readable storage medium. A person of ordinary skill in the art can understand that all or part of the steps in the method for implementing the above embodiment can be completed by instructing the processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid-state disk (SSD)), etc.

[0270] An embodiment of the present invention further provides an electronic device, which includes a processor and a memory.

[0271] The memory is used to store computer programs.

[0272] The memory includes: ROM, RAM, disk, USB flash drive, memory card or CD and other media that can store program codes.

[0273] The processor is connected to the memory and is used to execute the computer program stored in the memory so that the electronic device executes the above-mentioned intelligent complaint prediction method based on the investment advisory industry.

[0274] Preferably, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0275] like Figure 5 As shown, the electronic device of the present invention is in the form of a general computing device. The components of the electronic device may include but are not limited to: one or more processors or processing units 51, a memory 52, and a bus 53 connecting different system components (including the memory 52 and the processing unit 51).

[0276] Bus 53 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0277] Electronic devices typically include a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, removable and non-removable media.

[0278] The memory 52 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 521 and / or cache memory 522. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 523 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 53 via one or more data medium interfaces. The memory 52 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0279] A program / utility 524 having a set (at least one) of program modules 5241 may be stored, for example, in the memory 52, such program modules 5241 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination thereof may include an implementation of a network environment. The program modules 5241 generally perform the functions and / or methods of the embodiments described herein.

[0280] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, displays, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via input / output (I / O) interface 54. Furthermore, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 55. Figure 5 As shown, the network adapter 55 communicates with other modules of the electronic device via the bus 53. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0281] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. An intelligent complaint prediction method based on the investment consulting industry, characterized in that: The method comprises the following steps: Collect user data and pre-process it; Based on the preprocessed user data characteristics, construct feature engineering for complaint prediction; Based on the feature engineering, a first prediction sub-model, a second prediction sub-model and a third prediction sub-model are respectively constructed and trained respectively to obtain corresponding user complaint prediction results; The user complaint prediction results of each sub-model are integrated into the prediction parent model to obtain the user's final complaint prediction results.

2. The intelligent complaint prediction method based on the investment consulting industry according to claim 1 is characterized by: Constructing a first prediction sub-model based on the feature engineering and training it, and obtaining the corresponding user complaint prediction results include the following steps: Using a large language model as a pre-training basis for the first prediction sub-model; Setting a first business data set and a first business rule; Inputting the first business data set into the first prediction sub-model to obtain a first complaint analysis result; Training the first prediction sub-model using the first business rule based on the first complaint analysis result; Based on the trained first prediction sub-model, a first user complaint prediction result is obtained.

3. The intelligent complaint prediction method based on the investment consulting industry according to claim 1 is characterized by: Constructing a second prediction sub-model based on the feature engineering and training it, and obtaining the corresponding user complaint prediction results include the following steps: The BERT model is used as the pre-training basis for the second prediction sub-model; Setting a second business data set and a second business rule; Inputting the second business data set into the second prediction sub-model to obtain a second complaint analysis result; Based on the second complaint analysis result, the second prediction sub-model is trained using the second business rule; Based on the trained second prediction sub-model, a second user complaint prediction result is obtained.

4. The intelligent complaint prediction method based on the investment consulting industry according to claim 3 is characterized by: After obtaining the second user complaint prediction result based on the trained second prediction sub-model, the following steps are also included: Select a test dataset; Inputting the test data set into the second prediction sub-model, and obtaining corresponding test results through different strategies; Compare the test prediction results with the actual complaint results to obtain corresponding performance indicators; The second prediction sub-model is tuned according to the performance indicator.

5. The intelligent complaint prediction method based on the investment consulting industry according to claim 1 is characterized by: Constructing a third prediction sub-model based on the feature engineering and training it, and obtaining the corresponding user complaint prediction results include the following steps: Select the BERT model as the pre-trained model for the third prediction sub-model; Set the third business data set; Inputting the third business data set into the third prediction sub-model to obtain a third complaint analysis result; wherein the third analysis result includes user direct features and user indirect features; fusing the direct features and indirect features of the user data; Training the third prediction sub-model based on the fused features; Based on the trained third prediction sub-model, a third user complaint prediction result is obtained.

6. The intelligent complaint prediction method based on the investment consulting industry according to claim 1 is characterized by: Integrating the user complaint prediction results of each sub-model into the prediction master model to obtain the final user complaint prediction results includes the following steps: The user complaint prediction results of each sub-model are summarized based on a voting mechanism to obtain the final user complaint prediction result.

7. The intelligent complaint prediction method based on the investment consulting industry according to claim 1 is characterized by: Before integrating the user complaint prediction results of each sub-model into the prediction master model, the following steps are also included: Assign weights to the user complaint prediction results of each sub-model; The weighted average method is used to fuse the prediction results of each sub-model to obtain the fused prediction results.

8. An intelligent complaint prediction system based on the investment consulting industry, characterized in that: The system includes an acquisition and processing module, a feature construction module, a model construction module and a complaint prediction module; The collection and processing module is used to collect user data and perform preprocessing; The feature construction module is used to construct feature engineering for complaint prediction based on the preprocessed user data characteristics; The model building module is used to respectively build a first prediction sub-model, a second prediction sub-model and a third prediction sub-model based on the feature engineering and perform training to obtain corresponding user complaint prediction results; The complaint prediction module is used to integrate the user complaint prediction results of each sub-model into the prediction parent model to obtain the user's final complaint prediction result.

9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory so that the electronic device executes the intelligent complaint prediction method based on the investment advisory industry as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by an electronic device, the intelligent complaint prediction method based on the investment advisory industry described in any one of claims 1 to 7 is implemented.

Images