Method for training information recommendation model, information display processing method and related device

Abstract

The application provides a training method of an information recommendation model, an information display processing method and related devices. A service processing index score of each first service work order data is determined. First service work order data with a service processing index score meeting a preset processing index score is selected as second service work order data. First data of each training sample is determined according to each service work order information in each second service work order data. Second data of each training sample is determined according to each service processing result in each second service work order data. A training set is generated according to the first data of each training sample and the second data of each training sample. An original model is trained through the training set to obtain an information recommendation model. The information recommendation model can be trained by using more effective training samples, which is beneficial to improving the intelligence and functionality of service work order allocation.

Description

Technical Field

[0001] This application relates to the field of electronic digital data processing technology, and in particular to a training method for an information recommendation model, an information display processing method, and related apparatus. Background Technology

[0002] Currently, customer service dispatch systems have a work order recommendation model that recommends suitable work orders based on the individual work characteristics of the work order dispatchers. The model's input includes the basic information of the work order and the individual work characteristics of the dispatchers. The output is a matching score; the higher the matching score, the more suitable the current work order is for the dispatcher. After the work order allocation stage, the dispatchers process the work according to the recommended work orders. However, even with these model-recommended work orders, dispatchers still need to choose how to handle them. Therefore, further improving the intelligence and functionality of work order allocation remains a challenge. Summary of the Invention

[0003] In view of this, this application provides a training method, an information display processing method, and related apparatus for an information recommendation model. By selecting business work order data whose business processing index scores meet the preset processing index scores as training samples, the information recommendation model can be trained with more effective training samples, ensuring that the information recommendation model can recommend suitable information for different execution entities, which is conducive to improving the intelligence and functionality of business work order allocation.

[0004] In a first aspect, embodiments of this application provide a training method for an information recommendation model, applied to a server, the method comprising:

[0005] Determine the business processing indicator score for each first business work order data;

[0006] The first business work order data that meets the preset processing indicator score is selected as the second business work order data.

[0007] Based on the information of each business work order in each second business work order data, determine the first data of each training sample;

[0008] The second data for each training sample is determined based on the processing result of each business in each second business work order data.

[0009] A training set is generated based on the first data and the second data of each training sample;

[0010] The original model is trained using the training set to obtain the information recommendation model.

[0011] Secondly, embodiments of this application provide an information display processing method applied to a customer service user device, the method comprising:

[0012] The system receives a first message from the server. This first message represents the processing order of each pending business order from a customer service user. The processing order is represented by a work order sequence. The first message is determined by an information recommendation model, which is determined by the server in the following ways: determining a business processing indicator score for each first business order data; selecting first business order data whose business processing indicator scores match a preset processing indicator score as second business order data; determining first data for each training sample based on each business order information in each second business order data; determining second data for each training sample based on each business processing result in each second business order data; generating a training set based on the first data and the second data of each training sample; and training the original model using the training set to obtain the information recommendation model.

[0013] This displays the processing order for each pending business order.

[0014] Thirdly, embodiments of this application provide a training apparatus for an information recommendation model, applied to a server, the apparatus comprising:

[0015] The indicator determination unit is used to determine the business processing indicator score for each first business work order data.

[0016] The filtering unit is used to filter out the first business work order data whose business processing index score meets the preset processing index score, and use it as the second business work order data.

[0017] The first determining unit is used to determine the first data of each training sample based on the information of each business work order in each second business work order data.

[0018] The second determining unit is used to determine the second data of each training sample based on each business processing result in each second business work order data;

[0019] The training set determination unit is used to generate a training set based on the first data of each training sample and the second data of each training sample;

[0020] The training unit is used to train the original model using the training set to obtain the information recommendation model.

[0021] As can be seen, through the above-described training method, information display processing method, and related devices for the information recommendation model, firstly, the business processing index score for each first business work order data is determined; first business work order data whose business processing index scores meet the preset processing index scores are selected as second business work order data; based on each business work order information in each second business work order data, the first data for each training sample is determined; based on each business processing result in each second business work order data, the second data for each training sample is determined; a training set is generated based on the first data and the second data of each training sample; the original model is trained using the training set to obtain the information recommendation model. This allows the information recommendation model to be trained with more effective training samples, ensuring that the information recommendation model can recommend suitable information for different execution entities, which is beneficial to improving the intelligence and functionality of business work order allocation. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A system architecture diagram of a training method for an information recommendation model provided in an embodiment of this application;

[0024] Figure 2 A flowchart illustrating a training method for an information recommendation model provided in an embodiment of this application;

[0025] Figure 3 This is a schematic diagram of the structure of an information recommendation model provided in an embodiment of this application;

[0026] Figure 4 A flowchart illustrating an information display processing method provided in an embodiment of this application;

[0027] Figure 5 A schematic diagram of an information display interface provided in an embodiment of this application;

[0028] Figure 6 This application provides a schematic diagram of the structure of a server according to an embodiment of the present application.

[0029] Figure 7 This application provides a schematic diagram of the structure of a customer service user device.

[0030] Figure 8 A functional unit block diagram of a training device for an information recommendation model provided in an embodiment of this application;

[0031] Figure 9 This is a block diagram of the functional units of an information display processing device provided in an embodiment of this application. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0033] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0034] It should be understood that the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this document indicates that the preceding and following related objects are in an "or" relationship. In the embodiments of this application, "multiple" refers to two or more.

[0035] In the embodiments of this application, "at least one item" or its similar expression refers to any combination of these items, including any combination of a single item or a plurality of items. "One or more" means one or more, while "multiple" means two or more. For example, "at least one item" of a, b, or c can represent the following seven cases: a, b, c; a and b; a and c; b and c; a, b, and c. Each of a, b, and c can be an element or a set containing one or more elements.

[0036] In this application, the term "connection" refers to various connection methods, such as direct connection or indirect connection, to achieve communication between devices. This application does not impose any limitations on this.

[0037] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0038] The relevant terms used in the embodiments of this application are explained below.

[0039] Loan borrower: referred to as borrower or customer, refers to an enterprise or individual that borrows monetary funds from a lender by using its own credit or property as collateral, or by using a third party as collateral, in credit activities. In this embodiment of the application, the loan borrower can be understood as the party to be repaid.

[0040] Loan contact person: referred to as contact person, who helps contact the borrower when the lending institution is unable to do so. The contact person does not assume loan liability. In this embodiment, the target information of the party to be repaid can be the contact information of the loan contact person or the contact information of the loan borrower.

[0041] Lender: refers to a person or financial institution that uses credit funds or its own funds to issue loans to borrowers in lending activities. It generally refers to commercial banks, financial institutions, and central banks.

[0042] "Caller" refers to loan collection personnel in financial institutions. In this embodiment of the application, the collection party can be understood as a caller or a group of callers, i.e., customer service users.

[0043] Follow-up call: After communication, the customer made the payment.

[0044] No follow-up notice issued: After communication, the customer has not made payment and remains overdue.

[0045] Promise To Pay (PTP): When a customer promises to repay a certain amount of debt within a specified period through telephone collection, it is called a promise to pay. Here, it refers to the number of customers who promise to pay.

[0046] Kept Promise (KP): The number of customers who actually repay their loans after committing to do so in PTP.

[0047] The embodiments of this application can be applied to the following application scenarios, including but not limited to: neural network-based processing systems deployed on electronic devices, large-scale information recommendation (e.g., recalling targets from a large number of candidates, ranking, etc.), speech signal processing, natural language processing, recommendation systems, etc. The embodiments of this application can be adjusted and improved according to specific application environments, and are not specifically limited here.

[0048] The electronic device in this application embodiment may be a portable electronic device that also includes other functions such as a personal digital assistant and / or music player, such as a mobile phone, tablet computer, or wearable electronic device with wireless communication capabilities (such as a smartwatch). Exemplary embodiments of the portable electronic device include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device may also be other portable electronic devices, such as a laptop computer. It should also be understood that in some other embodiments, the aforementioned electronic device may not be a portable electronic device, but rather a desktop computer, server, etc.

[0049] For easier understanding, please refer to Figure 1 , Figure 1 This is a system architecture diagram of a training method for an information recommendation model provided in an embodiment of this application. (See diagram for example.) Figure 1 As shown, it includes a training set construction module 110 and a training module 120.

[0050] The training set construction module 110 is used to evaluate each business work order data and select business work order data whose business processing indicators meet the preset processing indicator scores to construct the training set. For ease of distinction, in this embodiment, the business work order data before screening is referred to as the first business work order data, and the selected business work order data is referred to as the second business work order data. The business work order data may include execution subject information, execution object information, execution process information, and basic business information, etc. The business work order data can be evaluated based on the above information to obtain the business processing indicator score for each business work order data. This score can reflect the conversion status, communication status, violation status, and complaint status of the business work order data, etc., which will not be elaborated here.

[0051] As can be seen, the training set construction module 110 described above can process a large amount of data, select data that is more suitable as a training set, improve the accuracy of the subsequent training information recommendation model, and thus improve the intelligence and functionality of business work order allocation.

[0052] The training module 120 is equipped with a cross-entropy loss function and / or a ranking loss function. It uses the business work order information in the training set to determine the input data and label data used to train the original model. It compares the output of the original model with the label data and uses the cross-entropy loss function and / or ranking loss function to perform reverse iteration on the original model until the original model converges to obtain the information recommendation model.

[0053] As can be seen, the original model can be updated using two loss functions through the training module 120. With the introduction of the ranking loss function, it can be ensured that the information recommendation model can recommend suitable information for different execution subjects. Furthermore, the adaptation here also introduces the time dimension, which is beneficial to improving the intelligence and functionality of business work order allocation.

[0054] To facilitate understanding, let's illustrate the debt collection process using a work order as an example. When a borrower defaults on a loan, collection efforts are required. The automatic allocation system distributes target information about borrowers to customer service user devices, who then use their devices to make calls to those borrowers. The automatic allocation system can incorporate an information recommendation model. This model can recommend relevant information about borrowers with a higher probability of repayment after being contacted by the current customer service user. Furthermore, the system can sort the target information based on repayment probability, allowing customer service users to make calls according to this ranking, thereby improving collection efficiency.

[0055] The following describes a training method for an information recommendation model in an embodiment of this application. (See also...) Figure 2 , Figure 2 The training method for an information recommendation model provided in this application embodiment, applied to a server, specifically includes the following steps:

[0056] Step 201: Determine the business processing indicator score for each first business work order data.

[0057] The first business work order data can come from the database and may include relevant information about completed business work orders.

[0058] Specifically, for each first business work order data, the following can be identified: first processing conversion data, first communication data, first violation data, and first complaint data; based on a first weight, the first processing conversion data and the first communication data corresponding to each first business work order data are determined to form a first processing indicator score; based on a second weight, the first violation data and the first complaint data corresponding to each first business work order data are determined to form a second processing indicator score; and based on the first processing indicator score and the second processing indicator score, the business processing indicator score for each first business work order data is determined.

[0059] Specifically, the first processing conversion data reflects the achievement of the work order's objectives; the first communication data reflects the communication quality of the customer service user during the handling of the work order; the first violation data reflects the violations committed by the customer service user during the handling of the work order; and the first complaint data reflects the number of complaints received by the customer service user during the handling of the work order. A higher score for the first processing indicator indicates that the work order's objectives are more closely aligned with the requirements, and the communication quality is more in line with those requirements. Conversely, a higher score for the second processing indicator indicates more serious violations and complaints committed by the customer service user.

[0060] For example, in a debt collection scenario, the first business order data that can be obtained includes each customer service user (i.e., candidate debt collector)'s historical debt collection amount data, historical communication quantity data, historical effective communication data, historical communication ability data, historical communication efficiency data, historical debt collection conversion data, historical violation data, and historical complaint data, etc.

[0061] In one possible embodiment, for historical collection amount data, the total monthly repayment amount e1 after collection by the candidate collection party can be calculated, wherein the total monthly repayment amount is positively correlated with collection ability.

[0062] In one possible implementation, for historical communication data, the average daily number of calls e2 for the current month can be calculated. To ensure stability, only calls with a duration longer than a preset duration can be counted. The average daily number of calls is positively correlated with collection capabilities.

[0063] In one possible implementation, for historical valid communication data, the average duration e3 of a single call can be calculated for each daily segment of the current month. Specifically, the average call duration from 8:00 AM to 12:00 PM can be determined. Average call duration from 2 PM to 8 PM Then set α is the weight, with a default value of 0.5. To ensure stability, only calls with a duration exceeding the preset duration can be counted. The average duration of a single call in each daily segment is positively correlated with collection capabilities.

[0064] In one possible implementation, for historical communication capability data, the average daily PTP number e4 for the current month can be calculated, which is the number of borrowers who promised to repay during the phone calls, wherein the average daily PTP number is positively correlated with collection capability.

[0065] In one possible implementation, for historical communication efficiency data, the total dialing time t for follow-up calls in the current month can be calculated first. c Total call time t sum Then, the historical communication efficiency e5 = t is obtained. c / tsum Among them, historical communication efficiency is positively correlated with collection ability.

[0066] In one possible implementation, for historical collection conversion data, the average daily PTP quantity N for the current month can be calculated. PTP With the number of KP N KP The formula for calculating historical debt collection conversion rate is: e6 = N KP / N PTP Among them, historical debt collection conversion rate is positively correlated with debt collection ability.

[0067] In one possible embodiment, for historical violation data, the number of violation words by the collection party or the number of calls containing violation words can be calculated in the voice-to-text data of all calls in the current month, and recorded as the historical violation count e7, wherein the historical violation count is inversely correlated with collection ability.

[0068] In one possible implementation, for historical complaint data, the number of complaints in the current month, e8, can be calculated, where the number of historical complaints is inversely correlated with collection ability.

[0069] Then, the collection capability is determined using the following formula:

[0070]

[0071] Where S is called the business processing indicator score, and E i With F i The normalized index is calculated using the following formula:

[0072]

[0073] Among them, e i,min With e i,max This represents the minimum and maximum values ​​achieved by all collection agencies in the department being evaluated under the i-th factor. i As a weighting factor for positive (improved collection capabilities) indicators, v i The weighting of negative (reduced collection ability) indicators.

[0074] As can be seen, this approach allows us to determine the processing status of the first business work order data from multiple dimensions, providing data support for subsequent screening.

[0075] Step 202: Select the first business work order data whose business processing index score meets the preset processing index score, and use it as the second business work order data.

[0076] Taking the above collection scenario as an example, the candidate collection party whose business processing indicator score S is greater than the preset processing indicator score can be identified as a customer service user.

[0077] In one possible embodiment, customer service users can be divided into multiple levels. Customer service users with higher levels have stronger collection capabilities. Since collection efficiency is higher when a collection party with strong collection capabilities chooses the party waiting to repay the debt, the multi-source data corresponding to the customer service users identified above can be used as training samples to train the original model. It can be understood that the multi-source data corresponding to the customer service users here is the second business work order data mentioned above.

[0078] As can be seen, this method can accurately determine the second business work order data, ensuring that the subsequently determined training samples are more effective for training the model.

[0079] Step 203: Determine the first data for each training sample based on the information of each business work order in each second business work order data.

[0080] The first data represents the input data used to train the original model.

[0081] Specifically, based on the information of each business work order in each second business work order data, the following can be determined for each second business work order: execution subject information, execution object information, execution process information, and basic business information; based on the following can be determined for each second business work order: execution subject information, execution object information, execution process information, and basic business information, the first data of each training sample can be determined.

[0082] Since there are many data sources, the first data can be divided into the first input data of each customer service user and the second input data of the business user corresponding to each customer service user. It should be noted that the customer service user serves the business user, and the customer service user is the subject of execution of the business work order, while the business user is the object of execution.

[0083] Taking debt collection as an example, the initial input data for each customer service user can include their basic personal characteristics (gender, age, years of service, group, workplace, etc.), call characteristics (average daily call time, average daily call start time, etc.), debt collection ability characteristics (number of calls made daily, daily promised repayment amount, daily achieved repayment amount, number of violations per day, number of complaints received per month, etc.), and historical conversation text characteristics. No specific limitations are set here.

[0084] The second input data for each business user may include the user's basic personal characteristics (gender, age, education, address, place of origin, years of employment, whether there are car loans / mortgages, etc.), call characteristics (average daily call time, number of historical calls, etc.), business characteristics (total loan amount, overdue amount, number of overdue days, number of contracts, maximum number of overdue days in 3 months, number of complaints in 3 months, etc.), historical dialogue text characteristics, and other derived characteristics (whether there are multiple loans), etc. No specific limitations are set here.

[0085] As can be seen, this can enrich the input data for training the model and improve the effect of subsequent model training.

[0086] Step 204: Determine the second data for each training sample based on the processing result of each business in each second business work order data.

[0087] The second data represents the label data used to train the original model.

[0088] Specifically, the business processing result of each second business work order can be determined based on the business work order information in each second business work order data; and the second data of each training sample can be determined based on the business processing result of each second business work order. This business processing result can be success or failure, or it can be a specific processing progress, etc., and is not specifically limited here.

[0089] Step 205: Generate a training set based on the first data and the second data of each training sample.

[0090] As can be seen, this method can yield more effective training samples for model training.

[0091] Step 206: Train the original model using the training set to obtain the information recommendation model.

[0092] Specifically, the first data of each training sample can be input into the original model to obtain the predicted probability that the business processing result of the business work order to which each training sample belongs is successful; a first loss function is determined based on each predicted probability and the business processing result in each training sample; the original model is subjected to reverse iterative processing through the first loss function until the original model converges to obtain the information recommendation model.

[0093] In one possible embodiment, after determining a first loss function based on each predicted probability and the business processing result in each training sample, it can be determined that in the business order information of each training sample, the first moment of execution corresponding to the second business order data when the business processing result is successful can be determined; the second moment of execution corresponding to the second business order data when the business processing result is unsuccessful can be determined; a second loss function is determined based on the first moment and the second moment; the original model is iteratively processed in reverse using the first loss function and the second loss function until the original model converges to obtain the information recommendation model.

[0094] The first loss function can be the cross-entropy loss function, and the second loss function can be the ranking loss function.

[0095] As can be seen, by introducing time-related parameters, we can not only recommend business orders with a higher probability of success after being processed by the current customer service user, but also determine which business orders are more suitable for early processing, greatly improving the intelligence and functionality of information recommendation.

[0096] In one possible embodiment, the structure of the information recommendation model is described. The model includes a first network module, a second network module, and an output module. The first network module is used for extracting low-order features, and the second network module is used for extracting high-order features. This is because multi-source data often suffers from missing features, exhibiting high dimensionality and sparsity. For this type of data, the focus is on learning combined features. Combined features include second-order, third-order, and even higher-order features; the higher the order, the more complex and difficult to learn. Both high-order and low-order combined features are crucial for modeling. The output module outputs the repayment probability.

[0097] To facilitate understanding, the following will be combined with Figure 3 An information recommendation model from one embodiment of this application is described by way of example. Figure 3 This is a schematic diagram of the structure of an information recommendation model provided in an embodiment of this application. The information recommendation model includes a sparse feature layer 310, a dense embedding layer 320, a first network layer 330, a second network layer 340, and an output layer 350.

[0098] Among them, the sparse feature layer 310 represents the concatenation of categorical features and numerical features after one-hot encoding. This is because the training data contains discrete data and continuous data. Discrete data needs to be transformed by one-hot encoding, while continuous data can be discretized first and then transformed by one-hot encoding.

[0099] The dense embedding layer 320 is used to embed high-dimensional sparse input data into low-dimensional dense vectors. However, the data input to the first network layer 330 and the data input to the second network layer 340 differ. After passing through the dense embedding layer 320, both the unembedded and embedded data can be used as inputs to the first network layer 330, and each dense vector can be horizontally concatenated as input to the second network layer 340. Different input data contain different features, and the embedding processes are independent of each other. In this embodiment, by default, the inputs to the first network layer 330 and the second network layer 340 contain the same features, both including all features.

[0100] The first network layer 330 can be a factorization machine, comprising a linear part and a cross part. The linear part performs pairwise multiplication of features, assigns weights to each feature, and then sums the results; this reflects first-order features. The cross part performs pairwise multiplication of features, assigns weights, and then sums the results; this reflects second-order combined features. The results from both parts are then summed to obtain the output of the first network layer 330. It should be noted that unembedded data is used as input to the first network layer 330 for first-order feature extraction, while embedded data is used as input for second-order feature extraction.

[0101] The second network layer 340 can be a deep neural network. The input of the second network layer 340 is the horizontal concatenation of all dense vectors. Then, after multiple hidden layers and non-linear transformations, the output of the second network layer 340 is obtained. It is generally mapped to 1 dimension because it needs to be accumulated with the result of the first network layer 330.

[0102] The output layer 350 can use an activation function to process the data obtained by adding the output data of the first network layer 330 and the output data of the second network layer 340 to obtain the repayment probability.

[0103] For example, let's illustrate the prediction process of an information recommendation model using a formula:

[0104] y dnn =DNN(Concat(Embedding(x)) dnn )00

[0105] y fm =FM(x fm Embedding(x fm ))

[0106]

[0107] Here, Embedding is an embedding operation that transforms high-dimensional sparse input data (vector) x dnn Embedding is performed to obtain low-dimensional dense vectors. Different input data contain different features, and the embedding processes are independent of each other. Concat means concatenation; each dense vector is concatenated horizontally and used as input to the second network layer 340 (DNN). Original (unembedded) input x fm The embedding results are used as input to the first network layer 330 (FM) for the extraction of first-order and second-order features, respectively. fm With x dnn Both represent input data, and they are allowed to contain different features. In this invention, by default, both contain the same features, i.e., all features, i.e., x.fm =x dnn =x. This represents the repayment probability predicted by the model.

[0108] The loss function in the embodiments of this application includes a first loss function, which may be a cross-entropy loss function.

[0109] Specifically, The predicted label is denoted as . Cross-entropy is used as the loss function for the classification task, which primarily identifies customers who have received or not received reminders. The calculation formula is as follows:

[0110]

[0111] In the formula, y represents the true label. C represents the number of categories, i.e., C = 2, and L is the number of samples.

[0112] The original model can be iteratively processed using the first loss function until it converges, resulting in the trained information recommendation model. This can improve the accuracy of the information recommendation model in predicting debtors who are prone to collection efforts.

[0113] In one possible embodiment, the loss function may further include a second loss function, which may be a ranking loss function.

[0114] Specifically, the following formula will be used to explain:

[0115]

[0116] Where M and N represent the number of samples predicted as having produced catalyst and those not, respectively, and t m With t n This represents the sum of the dial start times for samples predicted to receive a call and those not predicted to receive a call. The dial start time is an attribute of the call data. The ranking loss function L... pair This method prioritizes samples with earlier call times and fewer calls, placing those with later calls and fewer calls at higher times at the bottom. The call start time is numerical data and is not included in the input features of the information recommendation model; it is only used to calculate the ranking loss L. pair The start time is directly introduced. Similarly, when making predictions, it is not necessary to use the start time as input data.

[0117] The final loss function is the loss of the classification task. The loss L of the sorting task pair The sum of the two:

[0118]

[0119] In the formula, α is the adjustment parameter. This loss function is used to identify customers who are easy to contact and suitable for early calls.

[0120] As can be seen, this method can identify debtors who are more likely to require collection efforts, as well as debtors who are more suitable to be contacted in advance, greatly improving the collection efficiency of the collection party.

[0121] Specifically, after obtaining a trained information recommendation model, by having the model learn from the data of debt collectors with strong collection capabilities, it can subsequently recommend high-value customers to each level of debt collector, while selecting lower-level debt collectors for verification.

[0122] 4K data points were randomly selected from the EF gear positions as test data. Table 1 compares the top 50% of trigger activation rates after sorting. Table 2 compares the actual average start times of the top 50% of samples after sorting. The times in Table 2 have been converted to decimal.

[0123] Table 1 shows the changes in catalyst rate.

[0124]

[0125] Table 2. Changes in average start time

[0126]

[0127] As shown in Table 1, the information recommendation model effectively filters out debtors who are more likely to trigger collection efforts by a specific collection agency. Furthermore, Table 2 shows that debtors more likely to trigger collection efforts start their collection processes earlier, further demonstrating the good ranking effect of the information recommendation model. It effectively identifies debtors who are more likely to trigger collection efforts and are suitable for being contacted in advance.

[0128] In one possible embodiment, after training the original model using the training set to obtain the information recommendation model, the information recommendation model can be used to perform a first processing on the business order data to be processed to obtain first feature data, wherein the business order data to be processed includes business order information for each business order to be processed; the information recommendation model can then be used to perform a second processing on the business order data to be processed to obtain second feature data; the information recommendation model can then be used to process the first feature data and the second feature data to obtain the probability that the business processing result of each business order to be processed is successful; and the processing order of each business order to be processed can be determined based on the probability that the business processing result of each business order is successful.

[0129] For example, in a debt collection scenario, different numbers of customers are dynamically allocated to different debt collectors each day, meaning that different debt collectors correspond to different customer "number pools".

[0130] The data of the first collection agent and the first debtor can be input into a trained information recommendation model. Based on the output of the trained information recommendation model, a target set of debtors can be determined. The data of the first collection agent can include the collector's basic personal characteristics (gender, age, years of service, group, workplace, etc.), call characteristics (average daily call time, average daily call start time, etc.), collection ability characteristics (number of calls per day, daily promised repayment amount, daily actual repayment amount, number of violations per day, number of complaints per month, etc.), and historical dialogue text characteristics, etc. No specific limitations are set here.

[0131] The data for the first borrower can include each borrower's basic personal characteristics (gender, age, education, address, place of origin, years of employment, whether they have car loans / mortgages, etc.), call characteristics (average daily call time, number of historical calls, etc.), business characteristics (total loan amount, overdue amount, number of overdue days, number of contracts, maximum number of overdue days in 3 months, number of complaints in 3 months, etc.), historical conversation text characteristics, and other derived characteristics (whether they have multiple loans). No specific limitations are set here.

[0132] In this context, the target borrowers in the target borrower set are arranged in a target sequence, with the earlier a target borrower appears, the greater the probability of repayment after being contacted by the first collection agency. In one possible embodiment, the earlier a target borrower appears, the earlier it is suitable for collection by the first collection agency. The target borrowers can be a subset of the first borrowers.

[0133] Next, the target information of the debtors can be sent to the first collection agent in the order of the target sequence. This target information can include contact information, overdue duration, overdue amount, personal information, etc., without specific limitations. The calls can be made automatically to the target debtors in the order of the target sequence, or manually by the first collection agent; details will not be elaborated here.

[0134] In one possible embodiment, the information recommendation method can make predictions when the "number pool" is expanded in batches. It can make predictions once a day or in real time, without any specific limitation.

[0135] In one possible implementation, unreachable numbers enter a "pending dialing sequence" and are re-entered into the number pool after a preset time, such as 1 hour.

[0136] This application also provides an information display processing method, applied to customer service user equipment, see [link to relevant documentation]. Figure 4 , Figure 4 A flowchart illustrating an information display processing method provided in this application embodiment specifically includes the following steps:

[0137] Step 401: Receive the first message from the server.

[0138] The first message is used to characterize the processing order of each pending business order of a customer service user. The characterization of the processing order includes a work order sequence. The first message is determined by an information recommendation model, which is determined by the server in the following ways: determining the business processing indicator score for each first business work order data; selecting first business work order data whose business processing indicator scores meet the preset processing indicator scores as second business work order data; determining the first data for each training sample based on each business work order information in each second business work order data; determining the second data for each training sample based on each business processing result in each second business work order data; generating a training set based on the first data and the second data of each training sample; and training the original model using the training set to obtain the information recommendation model.

[0139] Step 402 displays the processing order of each pending business order.

[0140] For details, see Figure 5 , Figure 5 This is a schematic diagram of an information display interface provided in an embodiment of this application. The pending business work orders can be displayed in order of priority. The priority here is determined by the success rate of the pending business for the customer service user. With a more intelligent display method, the efficiency of the customer service user's business work order processing can be improved.

[0141] The following is combined with Figure 6 One type of server in the embodiments of this application will be described. Figure 6 This is a schematic diagram of the structure of a server according to an embodiment of this application. The server 600 includes a processor 601, a memory 602, and a communication bus 603 for connecting the processor 601 and the memory 602.

[0142] In some possible implementations, memory 602 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), which is used to store program code executed by server 600 and data transmitted.

[0143] In some possible implementations, server 600 also includes a communication interface for receiving and sending data.

[0144] In some possible implementations, processor 601 may be one or more central processing units (CPUs). If processor 601 is a central processing unit (CPU), the CPU may be a single-core CPU or a multi-core CPU.

[0145] In some possible implementations, the processing module 701 may be a baseband chip, a chip, a central processing unit (CPU), a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

[0146] In specific implementation, the processor 601 in the server 600 executes the program instructions 621 stored in the memory 602 to perform the following operations: determine the business processing index score for each first business work order data; filter out the first business work order data whose business processing index scores meet the preset processing index scores as second business work order data; determine the first data for each training sample based on each business work order information in each second business work order data; determine the second data for each training sample based on each business processing result in each second business work order data; generate a training set based on the first data and the second data of each training sample; and train the original model using the training set to obtain the information recommendation model.

[0147] As can be seen, through the above-described training method, information display processing method, and related devices for the information recommendation model, firstly, the business processing index score for each first business work order data is determined; first business work order data whose business processing index scores meet the preset processing index scores are selected as second business work order data; based on each business work order information in each second business work order data, the first data for each training sample is determined; based on each business processing result in each second business work order data, the second data for each training sample is determined; a training set is generated based on the first data and the second data of each training sample; the original model is trained using the training set to obtain the information recommendation model. This allows the information recommendation model to be trained with more effective training samples, ensuring that the information recommendation model can recommend suitable information for different execution entities, which is beneficial to improving the intelligence and functionality of business work order allocation.

[0148] It should be noted that the specific implementation of each operation can be described in the corresponding description of the method embodiments shown above. The server 600 can be used to execute the method embodiments of this application, and will not be described again here.

[0149] The following is combined with Figure 7 This application describes one type of customer service user equipment in an embodiment. Figure 7 This is a schematic diagram of the structure of a customer service user equipment according to an embodiment of this application. The customer service user equipment 700 includes a processing module 701, a storage module 702, and a communication bus module 703 for connecting the processing module 701 and the storage module 702.

[0150] In some possible implementations, the storage module 702 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), which is used to store program code executed by the customer service user equipment 700 and data transmitted.

[0151] In some possible implementations, the customer service user equipment 700 also includes a communication interface for receiving and sending data.

[0152] In some possible implementations, the processing module 701 may be one or more central processing units (CPUs). If the processing module 701 is a central processing unit (CPU), the central processing unit (CPU) may be a single-core central processing unit (CPU) or a multi-core central processing unit (CPU).

[0153] In some possible implementations, the processing module 701 may be a baseband chip, a chip, a central processing unit (CPU), a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

[0154] In specific implementation, the processing module 701 in the customer service user device 700 executes the program instructions 721 stored in the storage module 702 to perform the following operations: receiving a first message from the server, the first message representing the processing order of each pending business work order of the customer service user, the representation of the processing order including work order sequence, the first message being determined by an information recommendation model, the information recommendation model being determined by the server in the following ways: determining the business processing index score of each first business work order data; filtering out the first business work order data whose business processing index score meets the preset processing index score as second business work order data; determining the first data of each training sample based on each business work order information in each second business work order data; determining the second data of each training sample based on each business processing result in each second business work order data; generating a training set based on the first data and the second data of each training sample; training the original model through the training set to obtain the information recommendation model; and displaying the processing order of each pending business work order.

[0155] As can be seen, through the above-described training method, information display processing method, and related devices for the information recommendation model, firstly, the business processing index score for each first business work order data is determined; first business work order data whose business processing index scores meet the preset processing index scores are selected as second business work order data; based on each business work order information in each second business work order data, the first data for each training sample is determined; based on each business processing result in each second business work order data, the second data for each training sample is determined; a training set is generated based on the first data and the second data of each training sample; the original model is trained using the training set to obtain the information recommendation model. This allows the information recommendation model to be trained with more effective training samples, ensuring that the information recommendation model can recommend suitable information for different execution entities, which is beneficial to improving the intelligence and functionality of business work order allocation.

[0156] It should be noted that the specific implementation of each operation can be described in the corresponding description of the method embodiments shown above. The customer service user equipment 700 can be used to execute the above method embodiments of this application, and will not be described again here.

[0157] The above primarily describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the electronic device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0158] This application embodiment can divide the electronic device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0159] When dividing each function into modules according to its corresponding function. Figure 8 A functional unit block diagram of a training device for an information recommendation model provided in this application embodiment, applied to a server, the training device 800 for the information recommendation model includes:

[0160] The indicator determination unit 810 is used to determine the business processing indicator score for each first business work order data.

[0161] The filtering unit 820 is used to filter out the first business work order data whose business processing index score meets the preset processing index score, and use it as the second business work order data.

[0162] The first determining unit 830 is used to determine the first data of each training sample based on each business work order information in each second business work order data.

[0163] The second determining unit 840 is used to determine the second data of each training sample based on each business processing result in each second business work order data;

[0164] The training set determination unit 850 is used to generate a training set based on the first data of each training sample and the second data of each training sample;

[0165] Training unit 860 is used to train the original model using the training set to obtain the information recommendation model.

[0166] As can be seen, the training method and related apparatus of the above-mentioned information recommendation model firstly determine the business processing index score for each first business work order data; then, select the first business work order data whose business processing index scores meet the preset processing index scores as the second business work order data; based on each business work order information in each second business work order data, determine the first data for each training sample; based on each business processing result in each second business work order data, determine the second data for each training sample; generate a training set based on the first data and the second data of each training sample; and train the original model using the training set to obtain the information recommendation model. This allows the information recommendation model to be trained with more effective training samples, ensuring that the information recommendation model can recommend suitable information for different execution entities, which is beneficial to improving the intelligence and functionality of business work order allocation.

[0167] It should be noted that the specific implementation of each operation can be described in the corresponding description of the method embodiments shown above. The training device 800 of the information recommendation model can be used to execute the method embodiments of this application, and will not be described again here.

[0168] When dividing each function into modules according to its corresponding function. Figure 9 A functional unit block diagram of an information display processing device provided in this application embodiment, applied to customer service user equipment, the information display processing device 900 includes:

[0169] The receiving unit 910 is configured to receive a first message from the server. This first message characterizes the processing order of each pending business order from a customer service user. The characterization of the processing order includes an order sequence. The first message is determined by an information recommendation model, which is determined by the server in the following manner: determining a business processing indicator score for each first business order data; selecting first business order data whose business processing indicator scores match a preset processing indicator score as second business order data; determining first data for each training sample based on each business order information in each second business order data; determining second data for each training sample based on each business processing result in each second business order data; generating a training set based on the first data and the second data of each training sample; and training the original model using the training set to obtain the information recommendation model.

[0170] Display unit 920 is used to display the processing order of each pending business work order.

[0171] As can be seen, through the above-described training method, information display processing method, and related devices for the information recommendation model, firstly, the business processing index score for each first business work order data is determined; first business work order data whose business processing index scores meet the preset processing index scores are selected as second business work order data; based on each business work order information in each second business work order data, the first data for each training sample is determined; based on each business processing result in each second business work order data, the second data for each training sample is determined; a training set is generated based on the first data and the second data of each training sample; the original model is trained using the training set to obtain the information recommendation model. This allows the information recommendation model to be trained with more effective training samples, ensuring that the information recommendation model can recommend suitable information for different execution entities, which is beneficial to improving the intelligence and functionality of business work order allocation.

[0172] It should be noted that the specific implementation of each operation can be described in the corresponding description of the method embodiments shown above. The information display processing device 900 can be used to execute the method embodiments of this application, and will not be described again here.

[0173] This application also provides a chip, including a processor, a memory, and a computer program or instructions stored in the memory, wherein the processor executes the computer program or instructions to implement the steps described in the above method embodiments.

[0174] This application also provides a chip module, including a transceiver component and a chip. The chip includes a processor, a memory, and a computer program or instructions stored in the memory, wherein the processor executes the computer program or instructions to implement the steps described in the above method embodiments.

[0175] This application also provides a computer storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments, wherein the computer includes an electronic device.

[0176] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may include an electronic device.

[0177] It should be noted that, for the sake of simplicity, the above embodiments are all described as a series of actions. Those skilled in the art should understand that this application is not limited to the described order of actions, as some steps in the embodiments of this application can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions, steps, modules, or units involved are not necessarily essential to the embodiments of this application.

[0178] In the above embodiments, the descriptions of each embodiment in this application have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0179] The steps of the methods or algorithms described in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in RAM, flash memory, ROM, EPROM, electrically erasable programmable read-only memory (EEPROM), registers, hard disk, portable hard disk, read-only optical disk (CD-ROM), or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Furthermore, the ASIC can reside in a terminal device or management device. Alternatively, the processor and storage medium can exist as discrete components in the terminal device or management device.

[0180] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in the embodiments of this application can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0181] The modules / units included in the various devices and products described in the above embodiments can be software modules / units, hardware modules / units, or a combination of both. For example, for devices and products applied to or integrated into a chip, all modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits. For devices and products applied to or integrated into a chip module, all modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The implementation is achieved through a software program that runs on a processor integrated within the chip module. The remaining modules / units (if any) can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into terminal equipment, each of their modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components within the terminal equipment. Alternatively, at least some modules / units can be implemented using a software program that runs on a processor integrated within the terminal equipment, while the remaining modules / units (if any) can be implemented using hardware methods such as circuits.

[0182] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above descriptions are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A training method for an information recommendation model, characterized in that, Applied to a server, the method includes: Determine the business processing indicator score for each first business work order data; The first business work order data that meets the preset processing indicator score is selected as the second business work order data. Based on the information of each business work order in each second business work order data, determine the first data of each training sample; The second data for each training sample is determined based on the processing result of each business in each second business work order data. A training set is generated based on the first data and the second data of each training sample; The original model is trained using the training set to obtain the information recommendation model.

2. The method according to claim 1, characterized in that, The determination of the business processing indicator score for each first business work order data includes: The data corresponding to each first business work order is determined as follows: first processing conversion data, first communication data, first violation data, and first complaint data; Based on the first weight, the first processing conversion data and the first communication data corresponding to each first business work order data, the first processing indicator score corresponding to each first business work order data is determined; The second processing indicator score corresponding to each first business work order data is determined based on the second weight, the first violation data and the first complaint data corresponding to each first business work order data; The business processing index score for each first business work order data is determined based on the first processing index score and the second processing index score.

3. The method according to claim 1, characterized in that, The step of determining the first data for each training sample based on each business order information in each second business order data includes: Based on the information of each business work order in each second business work order data, determine the following for each second business work order: execution subject information, execution object information, execution process information, and basic business information; Based on the execution subject information, execution object information, execution process information, and basic business information of each second business work order, the first data of each training sample is determined.

4. The method according to claim 3, characterized in that, The step of determining the second data for each training sample based on each business processing result in each second business work order data includes: Based on the information of each business work order in each second business work order data, determine the business processing result of each second business work order; The second data for each training sample is determined based on the business processing result of each second business work order.

5. The method according to any one of claims 1-4, characterized in that, The first data represents the input data used to train the original model, and the second data represents the label data used to train the original model.

6. The method according to claim 1, characterized in that, The process of training the original model using the training set to obtain the information recommendation model includes: Input the first data of each training sample into the original model to obtain the predicted probability that the business processing result of the business work order to which each training sample belongs is successful. A first loss function is determined based on each predicted probability and the business processing result in each training sample; The original model is iterated in reverse using the first loss function until it converges, thus obtaining the information recommendation model.

7. The method according to claim 6, characterized in that, After determining the first loss function based on each predicted probability and the business processing result in each training sample, the method further includes: In each training sample, the business work order information is determined such that the business processing result is successful at the first moment of execution of the second business work order data. In each training sample, the second moment of execution processing corresponding to the second business order data is determined when the business processing result is a failure in the business order information. Determine the second loss function based on the first time point and the second time point; The original model is iterated backward using the first loss function and the second loss function until the original model converges, thus obtaining the information recommendation model.

8. The method according to claim 1, characterized in that, After training the original model using the training set to obtain the information recommendation model, the method further includes: The information recommendation model is used to perform a first processing on the business work order data to be processed to obtain first feature data. The business work order data to be processed includes the business work order information of each business work order to be processed. The information recommendation model is used to perform a second processing on the pending business work order data to obtain second feature data. The information recommendation model is used to process the first feature data and the second feature data to obtain the probability that the business processing result of each pending business work order is successful. The processing order of each pending business order is determined based on the probability that each business processing result is successful.

9. An information display processing method, characterized in that, Applied to customer service user equipment, the method includes: The system receives a first message from the server. This first message represents the processing order of each pending business order from a customer service user. The processing order is represented by a work order sequence. The first message is determined by an information recommendation model, which is determined by the server in the following ways: determining a business processing indicator score for each first business order data; selecting first business order data whose business processing indicator scores match a preset processing indicator score as second business order data; determining first data for each training sample based on each business order information in each second business order data; determining second data for each training sample based on each business processing result in each second business order data; generating a training set based on the first data and the second data of each training sample; and training the original model using the training set to obtain the information recommendation model. This displays the processing order for each pending business order.

10. A training device for an information recommendation model, characterized in that, Applied to a server, the device includes: The indicator determination unit is used to determine the business processing indicator score for each first business work order data. The filtering unit is used to filter out the first business work order data whose business processing index score meets the preset processing index score, and use it as the second business work order data. The first determining unit is used to determine the first data of each training sample based on the information of each business work order in each second business work order data. The second determining unit is used to determine the second data of each training sample based on each business processing result in each second business work order data; The training set determination unit is used to generate a training set based on the first data of each training sample and the second data of each training sample; The training unit is used to train the original model using the training set to obtain the information recommendation model.

Images