Method and related device for intention prediction and intention prediction model training

Abstract

The invention provides intention prediction and intention prediction model training methods and related devices, relates to the field of artificial intelligence and the technical field of data processing, and aims to improve the accuracy of object loss probability prediction. The method comprises the steps of obtaining a target feature vector of a target object based on a behavior feature vector and a preference feature vector of the target object; and the intention of the target object is predicted based on the target feature vector to obtain the intention probability of the target object, and the intention probability is used for representing the loss probability of the target object. When the target feature vector is determined, not only is the behavior feature vector of the object considered, but also the preference feature vector is added, the preference feature vector can better represent the behavior preference degree of the object, namely, the personalized feature of the object is reflected, so that the obtained target feature vector can better represent the importance degree of different behavior features to the target object, and the user experience is improved. And the accuracy of predicting the loss probability is improved.

Description

Background Art

[0002] With the continuous development of science and technology, a large number of people have participated in Internet life. With the peak of mobile Internet traffic, the growth of the number of new objects is limited. How to serve existing objects well and prevent object loss is the top priority of each target application.

[0003] In order to prevent object loss, in addition to improving the quality of the application itself, predicting whether an object is likely to be lost and taking corresponding retention measures for the object that is likely to be lost is also an effective way.

[0004] Usually, when predicting whether an object is likely to churn, the churn probability of the object is predicted mainly based on the object's recent behavioral characteristics in the target application. However, such a churn probability prediction method can only predict the churn probability based on the common characteristics of the objects that may churn, and cannot perform personalized churn probability prediction, resulting in low accuracy in predicting churn probability.

[0005] Therefore, a new method for predicting the probability of object churn is urgently needed to improve the accuracy of object churn probability prediction. Summary of the invention

[0006] The present application provides a method and related device for intent prediction and intent prediction model training to improve the accuracy of object churn probability prediction.

[0007] In a first aspect, the present application provides a method for intention prediction, the method comprising:

[0008] Based on the behavior feature vector and the preference feature vector of the target object, a target feature vector of the target object is obtained;

[0009] The intention of the target object is predicted based on the target feature vector to obtain the intention probability of the target object. The intention probability is used to indicate the probability of loss of the target object.

[0010] In the present application, when predicting the probability of target object loss, the target feature vector of the target object is first obtained based on the behavioral feature vector and preference feature vector of the target object; it can be seen that when determining the target feature vector, not only the behavioral feature vector of the object is taken into account, but also the preference feature vector of the object is added. The preference feature vector can better represent the degree of behavioral preference of the object, that is, it reflects the personalized characteristics (or essential characteristics) of the object, so the obtained target feature vector can better represent the importance of different behavioral characteristics to the target object; further, based on the target feature vector, the intention of the target object is predicted, and the intention probability used to represent the probability of target object flow is obtained. The importance of different behavioral characteristics to the target object can be learned, the personalized fitting of the target object is enhanced, and the accuracy of predicting the probability of loss is improved.

[0011] In a second aspect, an embodiment of the present application provides a method for training an intent prediction model, the method comprising:

[0012] Determine a training feature set of the training object according to the loss time of the training object and the behavior data of the training object, wherein the training feature set includes a label of the training object;

[0013] The training feature set of the training object and the identifier of the training object are concatenated to obtain a training sample;

[0014] Input the training samples into the intention prediction model and output the loss prediction value of the training object;

[0015] The intent prediction model is trained based on the difference between the churn prediction value of the training object and the label of the training object.

[0016] In the present application, when training the intention prediction model, the training feature set of the training object is first determined based on the loss time of the training object and the behavioral data of the training object, and the training feature set includes the label of the training object; then the training feature set of the training object and the identifier of the training object are spliced ​​to obtain a training sample. It can be seen that in addition to the behavioral data of the training object, the training sample also includes the identifier of the training object. The identifier of each training object corresponds to the preference data of the corresponding object, which represents the personalized characteristics (or essential characteristics) of the corresponding object; therefore, when training the intention prediction model based on the training sample, the preference data of the object identifier that best represents the essential characteristics of the object is added for training, which can explicitly learn the importance of different features to different objects, enhance the personalized fitting ability of the model, and thereby improve the accuracy of the model in predicting the probability of intent.

[0017] In a third aspect, the present application provides a device for intention prediction, the device comprising:

[0018] an obtaining unit, used for obtaining a target feature vector of the target object based on the behavior feature vector and the preference feature vector of the target object;

[0019] The prediction unit is used to predict the intention of the target object based on the target feature vector to obtain the intention probability of the target object, and the intention probability is used to represent the probability of loss of the target object.

[0020] In a possible implementation manner, the obtaining unit is specifically used to:

[0021] Through the intention prediction model, the target feature vector of the target object is obtained based on the behavior feature vector and preference feature vector of the target object.

[0022] In a possible implementation, the intent prediction model includes a feature extraction network, a gate network, and a fusion network. Through the intent prediction model, the acquisition unit is specifically used for:

[0023] Through the feature extraction network, feature extraction is performed on the behavior semantic vector associated with the behavior data of the target object to obtain the behavior feature vector of the target object;

[0024] Through the gate network, feature extraction is performed on the preference semantic vector associated with the target object's identification to obtain the preference feature vector of the target object. The dimension of the preference feature vector is the same as that of the behavior feature vector.

[0025] Through the fusion network, based on the first element in the preference feature vector, the second element in the behavior feature vector with the same position information as the first element is fused to obtain the target feature vector.

[0026] In a possible implementation, the gate network includes: a first fully connected layer and a second fully connected layer; the obtaining unit is specifically used for:

[0027] Through the first fully connected layer, the preference semantic vector associated with the target object's identification is transformed into a nonlinear feature vector to obtain the initial preference vector;

[0028] Through the second fully connected layer, the dimension of the initial preference vector is transformed to obtain the preference feature vector.

[0029] In a possible implementation manner, the obtaining unit is specifically used to:

[0030] The first element in the preference feature vector is used as the preference weight of the second element in the behavior feature vector that has the same position information as the first element;

[0031] weighting the corresponding second element in the behavior feature vector based on the preference weight to obtain a weighted behavior feature vector;

[0032] The weighted behavior feature vector is used as the target feature vector.

[0033] In a possible implementation, the intent prediction model further includes a fully connected layer, and the prediction unit is specifically used for:

[0034] Through the fully connected layer, the behavior of the target object is predicted based on the target feature vector to obtain the intention probability of the target object.

[0035] In a fourth aspect, an embodiment of the present application provides a device for training an intent prediction model, the device comprising:

[0036] A determination unit, configured to determine a training feature set of the training object according to the loss time of the training object and the behavior data of the training object, wherein the training feature set includes a label of the training object;

[0037] A concatenation unit, used for concatenating a training feature set of a training object and an identifier of the training object to obtain a training sample;

[0038] An input unit, used to input training samples into the intention prediction model and output a loss prediction value of the training object;

[0039] The training unit is used to train the intent prediction model based on a difference value between a churn prediction value of the training object and a label of the training object.

[0040] In a possible implementation, the intent prediction model includes a feature extraction network, a gate network, a fusion network, and a fully connected layer; the input unit is specifically used for:

[0041] Through the feature extraction network, feature extraction is performed on the behavior semantic vector associated with the behavior data of the training object to obtain the historical behavior feature vector;

[0042] Through the gate network, feature extraction is performed on the preference semantic vector associated with the identification of the training object to obtain the historical preference feature vector of the training object. The dimension of the historical preference feature vector is the same as that of the historical behavior feature vector.

[0043] Through the fusion network, based on the first element in the historical preference feature vector, the second element in the historical behavior feature vector with the same position information as the first element is fused to obtain the historical target feature vector;

[0044] Through the fully connected layer, the historical target feature vector is reduced in dimension to obtain the churn prediction value of the training object.

[0045] In a possible implementation, the gate network includes: a first fully connected layer and a second fully connected layer, and the input unit is specifically used for:

[0046] Through the first fully connected layer, the preference semantic vector associated with the identification of the training object is transformed into a nonlinear feature vector to obtain the initial vector of historical preference;

[0047] Through the second fully connected layer, the dimension of the historical preference initial vector is transformed to obtain the historical preference feature vector.

[0048] In a fifth aspect, the present application provides an electronic device comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor implements the steps of any of the above methods.

[0049] In a sixth aspect, the present application also provides a computer-readable storage medium, which includes a computer program. When the computer program runs on an electronic device, the computer program is used to enable the electronic device to execute the steps of any of the above methods.

[0050] In a seventh aspect, the present application also provides a computer program product, including a computer program, which implements the steps of any of the above methods when the computer program is executed by a processor.

[0051] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A schematic diagram of a possible application scenario provided for an embodiment of the present application;

[0053] Figure 2 A basic structure diagram of an intention prediction model provided in an embodiment of the present application;

[0054] Figure 3A A specific structural diagram of the first intention prediction model provided in the embodiment of the present application;

[0055] Figure 3B A specific structural diagram of the second intention prediction model provided in the embodiment of the present application;

[0056] Figure 3C A specific structural diagram of the third intention prediction model provided in the embodiment of the present application;

[0057] Figure 4 A flow chart of a method for constructing training samples provided in an embodiment of the present application;

[0058] Figure 5A A schematic diagram of adjusting training weights provided in an embodiment of the present application;

[0059] Figure 5B A schematic diagram of another method for adjusting training weights provided in an embodiment of the present application;

[0060] Figure 5C A schematic diagram of another method for adjusting training weights provided in an embodiment of the present application;

[0061] Figure 6 A flow chart of a method for training an intent prediction model provided in an embodiment of the present application;

[0062] Figure 7 A schematic diagram of an intention prediction model training provided in an embodiment of the present application;

[0063] Figure 8 A flow chart of a method for intention prediction provided in an embodiment of the present application;

[0064] Fig. 9 A schematic diagram of a first time range provided in an embodiment of the present application;

[0065] Fig.10 A schematic diagram of obtaining the probability of intention through an intention prediction model provided in an embodiment of the present application;

[0066] Fig.11 A structural diagram of an intention prediction device provided in an embodiment of the present application;

[0067] Fig.12 A structural diagram of a device for training an intention prediction model provided in an embodiment of the present application;

[0068] Fig.13 A hardware structure diagram of an electronic device in an embodiment of the present application;

[0069] Fig.14 This is a hardware structure diagram of another electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0071] It is understandable that in the following specific implementations of the present application, when the relevant data used to predict the probability of object loss is involved, when the various embodiments of the present application are applied to specific products or technologies, relevant licenses or consents need to be obtained, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, when it is necessary to obtain relevant behavioral data and preference data, relevant volunteers can be recruited and relevant agreements for volunteer authorization data can be signed, and then the data of these volunteers can be used for implementation; or, by implementing within the scope of the authorized organization, the following implementation method is implemented by using the data of internal members of the organization to identify internal members; or, the relevant data used in the specific implementation are all simulated data, such as simulated data generated in a simulated scenario.

[0072] The word “exemplary” is used hereinafter to mean “serving as an example, example, or illustration.” Any embodiment described as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0073] The terms "first" and "second" in this document are used for descriptive purposes only and should not be understood as explicitly or implicitly indicating relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of this application, unless otherwise specified, the meaning of "multiple" is two or more.

[0074] The following is a brief introduction to the design concept of the embodiment of the present application:

[0075] As mobile Internet traffic reaches its peak, the growth of new objects is limited. How to serve existing objects well and prevent object loss is a key issue facing each target application.

[0076] In terms of preventing object loss, in addition to improving the service quality of the application itself, the target application can also predict the probability of object loss, and then execute corresponding recovery strategies for the object based on the probability of object loss, so as to avoid object loss.

[0077] When predicting the probability of churn of an object, the model is usually used to determine whether the object is likely to churn based on the object's recent behavioral characteristics in the target application. For example, a logistic regression model is used to predict the probability of churn of an object based on the object's recent behavioral characteristics in the target application. Among them, the logistic regression model is actually a classification model, which has the advantages of a simple model and strong interpretability, but the logistic regression model has insufficient expressiveness and generalization capabilities and is not suitable for large-scale application scenarios; and when making predictions based on behavioral characteristics, the churn probability of different types of objects can only be roughly predicted based on the common characteristics that the churned objects may have, and personalized predictions cannot be made for the objects, resulting in low accuracy in predicting the probability of churn of objects.

[0078] In view of this, the embodiment of the present application provides a method and related device for intention prediction and intention prediction model training, which relates to the field of artificial intelligence and the field of data processing technology. In the embodiment of the present application, the intention prediction model is trained first, and then the intention prediction is performed through the intention prediction model obtained by the training.

[0079] When training the intention prediction model, first, the training feature set of the training object is determined according to the loss time of the training object and the behavioral data of the training object, and the training feature set includes the label of the training object; then the training feature set of the training object and the identifier of the training object are spliced ​​to obtain the training sample. Therefore, in addition to the behavioral data of the training object, the training sample also includes the identifier of the training object. The identifier of each training object corresponds to the preference data of the corresponding object, which represents the personalized characteristics (or essential characteristics) of the corresponding object; and then, when the preference data of the object identifier that best represents the essential characteristics of the object is added for training, the importance of different features to different objects can be explicitly learned, the personalized fitting ability of the model is enhanced, and the accuracy of the model in predicting the probability of intent is improved.

[0080] When the intention prediction model is used to predict the intention, the target feature vector of the target object is obtained based on the behavior feature vector and preference feature vector of the target object. At this time, not only the behavior feature vector of the object is taken into account, but also the preference feature vector of the object is added. The preference feature vector can better represent the degree of behavior preference of the object, that is, it reflects the personalized characteristics (or essential characteristics) of the object. Therefore, the obtained target feature vector can better represent the importance of different behavior characteristics to the target object. Furthermore, when the intention of the target object is predicted based on the target feature vector and the intention probability used to represent the probability of the target object flow is obtained, the importance of different behavior characteristics to the target object can be learned, the personalized fitting of the target object is enhanced, and the accuracy of predicting the churn probability is improved.

[0081] In an embodiment of the present application, after completing the prediction of the churn probability for the target object, when the churn probability of the target object is high, a corresponding recovery strategy can be formulated based on the preference characteristics of the target object, and then corresponding recovery information can be sent to the target object, thereby improving the recovery rate for different objects.

[0082] In an embodiment of the present application, the above-mentioned intention prediction process is performed through an intention prediction model, and the part of obtaining the intention prediction model and adjusting the model parameters involves artificial intelligence (AI) and machine learning (ML) technology, which is designed based on classification technology, feature extraction and machine learning in artificial intelligence.

[0083] Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0084] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, large-scale intent prediction technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, pre-trained models are also called large models and basic models. After fine-tuning, they can be widely used in downstream tasks in various major directions of artificial intelligence. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0085] Machine learning is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning and other technologies.

[0086] The following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0087] See also Figure 1 , is a schematic diagram of a possible application scenario provided in an embodiment of the present application, in which a terminal device 101 and a server 102 may be included.

[0088] The terminal device 101 includes but is not limited to mobile phones, tablet computers (PADs), personal computers (PCs), wearable devices, laptops, desktop computers, e-book readers, intelligent voice interaction devices, smart home appliances, car terminals and other devices; various clients can be installed on the terminal device, which can support online applications and applications with content browsing functions, or can be web pages, applets, etc.

[0089] Server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence applications.

[0090] The server 102 may include one or more processors 1021, a memory 1022, and an I / O interface 1023 for interacting with a terminal. In addition, the server 102 may also be configured with a database 1024, which may be used to store web page content information such as article content and image content. Among them, the memory 1022 of the server 102 may also store program instructions for the method of intent prediction and intent prediction model training provided in the embodiment of the present application. When these program instructions are executed by the processor 1021, they can be used to implement the steps of intent prediction provided in the embodiment of the present application to achieve the prediction of the probability of object loss, and can also implement the steps of intent prediction model training provided in the embodiment of the present application to obtain an intent prediction model with accurate output results.

[0091] The terminal device 101 and the server 102 may be directly or indirectly connected to each other through one or more communication networks 103. The communication network 103 may be a wired network or a wireless network, for example, a mobile cellular network or a wireless fidelity (Wireless-Fidelity, WIFI) network, or other possible networks, which are not limited in the embodiments of the present application.

[0092] It should be noted that each method in the embodiments of the present application can be executed by an electronic device, which may be a terminal device 101 or a server 102, that is, each method can be executed individually by the terminal device 101 or the server 102, or can be executed jointly by the terminal device 101 and the server 102. Which electronic device executes which step in the method for intent prediction and intent prediction model training provided in the embodiments of the present application can be set and adjusted according to actual conditions.

[0093] It should be noted that Figure 1 What is shown is just an example. In fact, the number and communication mode of terminal devices and servers are not limited and are not specifically limited in the embodiments of the present application.

[0094] To further illustrate the technical solution provided by the embodiment of the present application, the following takes the execution of the server alone as an example, and describes the method for training the intent prediction model provided by the exemplary implementation of the present application, and the method for intent prediction in conjunction with the accompanying drawings. It should be noted that the above application scenario is only shown for the sake of facilitating the understanding of the spirit and principle of the present application, and the implementation of the present application is not subject to any restrictions in this regard. In addition, although the operation of the method of the present application is described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution.

[0095] In order to ensure the accuracy of the intention prediction model, the embodiment of the present application provides a method for training the intention prediction model. At the same time, in order to verify the accuracy of the object loss probability obtained when performing intention prediction based on the intention prediction model, the embodiment of the present application proposes a model application method after obtaining the intention prediction model, specifically: using the intention prediction model, based on the behavior feature vector and preference feature vector of the target object, the intention probability of the target object is obtained, and the intention probability is used to represent the probability of loss of the target object.

[0096] Below, the overall implementation method of the embodiment of the present application is explained from the model training process and the model application process respectively.

[0097] Example 1: Model training process

[0098] In the embodiment of the present application, the training process is a process of training the intent prediction model using training samples, which may mainly include a model design phase, a training sample construction phase and a model training phase. Each phase will be introduced separately below.

[0099] 1. Model design stage

[0100] See also Figure 2 , Figure 2 A basic structure diagram of an intention prediction model provided in an embodiment of the present application, the intention prediction model includes: an embedding layer, at least one feature processing layer and a fully connected layer; wherein:

[0101] The embedding layer is used to convert high-dimensional data such as the historical behavior data of the training objects associated with the training samples and the historical preference data obtained based on the identification of the training objects into low-dimensional historical behavior semantic vectors and historical preference semantic vectors; it should be noted that the historical behavior semantic vectors and historical preference semantic vectors are shallow network feature vectors.

[0102] In an embodiment of the present application, historical behavior data and historical preference data are text description data; historical behavior data include but are not limited to: object portrait data of the training object, behavioral statistics data of the training object in the target application, real-time behavior data of the training object in the target application, and context data of the training object; wherein, the object portrait data include but are not limited to: gender, age, and city; context data include but are not limited to: the number of logins to the target application in the last N days, the number of clicks, the number of comments, and the number of previews in the last N days, where N is a positive integer.

[0103] At least one feature processing layer is used to obtain a historical target feature vector of the training object based on the historical behavior semantic vector and the historical preference semantic vector of the training object.

[0104] The fully connected layer is used to predict the intention of the training object based on the historical target feature vector of the training object, that is, to reduce the dimension of the historical target feature vector to obtain the churn prediction value of the training object.

[0105] In a possible implementation, each of the at least one feature processing layer includes a feature extraction network, a gate network, and a fusion network, and an output result of the feature extraction network has the same dimension as an output result of the gate network.

[0106] For example, the intention prediction model includes two feature processing layers, see Figure 3A , Figure 3A A specific structural diagram of the first intention prediction model provided in the embodiment of the present application; Figure 3A As can be seen from the figure, each feature processing layer includes a feature extraction network, a gate network, and a fusion network. Figure 3A When processing the model structure shown:

[0107] First, the object portrait data of the training object, the behavioral statistics data of the training object in the target application, the real-time behavioral data of the training object in the target application, and the context data of the training object are converted and processed respectively through the embedding layer to obtain the corresponding behavioral semantic vectors, and then the obtained behavioral semantic vectors are spliced ​​to obtain the historical behavioral semantic vectors of the behavioral data; similarly, the historical preference data determined based on the identification of the training object is converted and processed through the embedding layer to obtain the corresponding historical preference semantic vectors;

[0108] Next, the historical behavior semantic vector and the historical preference semantic vector are input into the first feature processing layer. In the first feature processing layer: first, the historical behavior semantic vector is feature extracted through the feature extraction network to obtain the corresponding historical behavior feature vector, and the historical preference semantic vector is feature extracted through the gate network to obtain the corresponding historical preference feature vector. The dimension of the historical behavior feature vector is the same as that of the historical preference feature vector. Then, based on the first element of the historical preference feature vector, the second element in the historical behavior feature vector with the same position information as the first element is fused through the fusion network to obtain the historical initial feature vector. It should be noted that the historical behavior feature vector and the historical preference feature vector are deep network feature vectors.

[0109] Then, the historical initial feature vector and the historical preference semantic vector are input into the second feature processing layer. In the second feature processing layer: first, the historical initial feature vector is feature extracted through the feature extraction network to obtain the corresponding historical behavior feature vector, and the historical preference semantic vector is feature extracted through the gate network to obtain the corresponding historical preference feature vector. The dimension of the historical behavior feature vector is the same as that of the historical preference feature vector; then, through the fusion network, based on the first element in the historical preference feature vector, the second element in the historical behavior feature vector associated with the initial feature vector with the same position information as the first element is fused to obtain the historical target feature vector;

[0110] Finally, the historical target feature vector is reduced in dimension through the fully connected layer to obtain the churn prediction value of the training object.

[0111] In a possible implementation, when there are multiple feature processing layers, there are feature processing layers that only include a feature extraction network, and there are feature extraction layers that include a feature extraction network, a gate network, and a fusion network.

[0112] For example, the intention prediction model includes two feature processing layers, see Figure 3B , Figure 3B A specific structural diagram of the second intention prediction model provided in the embodiment of the present application; Figure 3BIt can be seen that the first feature processing layer only includes the feature extraction network, and the second feature processing layer includes the feature extraction network, the gate network and the fusion network; at this time, based on Figure 3B When processing the model structure shown:

[0113] First, the object portrait data of the training object, the behavioral statistics data of the training object in the target application, the real-time behavioral data of the training object in the target application, and the context data of the training object are converted and processed respectively through the embedding layer to obtain the corresponding shallow feature vectors, and then the obtained shallow feature vectors are spliced ​​to obtain the historical behavior semantic vector; similarly, the historical preference data determined based on the identification of the training object is converted and processed through the embedding layer to obtain the corresponding shallow feature vector, and the shallow feature vector is used as the historical preference semantic vector;

[0114] Next, the historical behavior semantic vector is input into the first feature processing layer, in which: the historical behavior semantic vector is subjected to feature extraction through a feature extraction network to obtain a corresponding historical initial feature vector;

[0115] Then, the historical initial feature vector and the historical preference semantic vector output by the first feature processing layer are input into the second feature processing layer. In the second feature processing layer: first, feature extraction is performed on the historical initial feature vector through the feature extraction network to obtain the corresponding historical behavior feature vector, and feature extraction is performed on the historical preference semantic vector through the gate network to obtain the corresponding historical preference feature vector. The dimension of the historical behavior feature vector is the same as that of the historical preference semantic vector; then, based on the first element in the historical preference feature vector, the second element in the historical behavior feature vector with the same position information as the first element is fused through the fusion network to obtain the historical target feature vector;

[0116] Finally, the historical target feature vector is reduced in dimension through the fully connected layer to obtain the churn prediction value of the training object.

[0117] Another exemplary example is to take the intention prediction model including two feature processing layers as an example, see Figure 3C , Figure 3C A specific structural diagram of the third intention prediction model provided in the embodiment of the present application; Figure 3C It can be seen that the first feature processing layer includes feature extraction network, gate network and fusion network, and the second feature processing layer only includes feature extraction network; at this time, based on Figure 3C The processing method for the model structure shown can be referred to Figure 3B The method will not be repeated here.

[0118] In a possible implementation, the gate network includes: a first fully connected layer and a second fully connected layer, for example, see FIG. 3A to FIG. 3C The model structure; at this time, when the gate network is used to extract the historical preference semantic vector associated with the identification of the training object:

[0119] First, the behavior semantic vector associated with the identification of the training object is transformed into a nonlinear feature through the first fully connected layer to obtain the initial vector of historical preference. The activation function in the first fully connected layer is the Relu activation function, which is used to perform a corrected linear processing on the behavior semantic vector, that is, the values ​​greater than 0 in the behavior semantic vector are retained, and the values ​​not greater than 0 are corrected to 0.

[0120] Then, through the second fully connected layer, the dimension of the initial historical preference vector is transformed to obtain the historical preference feature vector; wherein, the activation function in the second fully connected layer is the Simgoid activation function, and the Simgoid activation function is used to normalize the behavior semantic vector, that is, to map each value in the behavior semantic vector to between (0, 1).

[0121] In one possible implementation, through a fusion network, based on the first element in the historical preference feature vector, the second element in the historical behavior feature vector with the same position information as the first element is fused, and when the historical target feature vector is obtained, the first element in the historical preference feature vector is used as the preference weight of the second element in the historical behavior feature vector with the same position information as the first element; based on the preference weight, the corresponding second element in the historical behavior feature vector is weighted to obtain a weighted historical behavior feature vector; the weighted historical behavior feature vector is used as the historical target feature vector. Among them, each first element in the historical preference feature vector occupies a position in the historical preference feature vector, and represents the degree of preference of the training object for the behavior type associated with the corresponding position, that is, the first element is a value; similarly, each second element in the historical behavior feature vector occupies a position in the historical behavior feature, and represents the degree of operation of the training on the behavior type associated with the corresponding position;

[0122] In the embodiment of the present application, the vector dimensions of the historical preference feature vector and the historical behavior feature vector are the same, and the behavior types associated with the same position are the same. Exemplarily, the historical preference feature vector and the historical behavior feature vector are both 2x2, and the behavior types associated with the (0,0) position are both active period features; the behavior types can be divided and set according to the different behavior types contained in the historical data, or they can be set by manually uploading different behavior types, and the present application does not limit this.

[0123] Since different objects have different behavior logic and behavior characteristics, the feature weights of the same position in the preference feature vector obtained for different objects are also different. For example, some objects will pay more attention to the quality of the content provided by the application, while some objects will pay more attention to the quality of the service provided by the application, or some objects will have active login behavior every day, while some objects will log in to the application intermittently and use certain services in the application. These different behavioral characteristics can be reflected in the feature weights corresponding to the preference feature vector. These feature weights are superimposed on the corresponding behavior feature vector to enhance the personalized fitting degree of the obtained object's churn characteristics.

[0124] In this application, a gate network is designed in the intention prediction model, which can make full use of the object-specific preference feature vector, and use the preference feature vector and the behavior feature vector for fusion processing. It can explicitly learn the importance of different features and is suitable for scenarios with large-scale objects. Therefore, using the intention prediction model provided in this application can improve the expressiveness and generalization capabilities.

[0125] 2. Training Sample Construction Phase

[0126] Training sample construction is of paramount importance in machine learning, and can be said to be the most important link. The training sample construction stage of the embodiment of the present application mainly includes: determining the training feature set of the training object, determining the identification of the training object, wherein the training feature set includes the historical behavior data of the training object and the label of the training object.

[0127] In a possible implementation, there are two cases of labels, one is a positive sample and the other is a negative sample. With respect to the training process of the intent prediction model provided in the embodiment of the present application, in the present application, a positive sample indicates that its corresponding training sample is a training sample of a lost object, and a negative sample indicates that its corresponding training sample is a training sample of a non-lost object.

[0128] For a running target application, the number of negative samples (i.e., training samples of non-lost objects) that can be obtained is much larger than the number of positive samples. Therefore, in order to avoid the features of a large number of negative samples completely covering the features of positive samples, the embodiment of the present application proposes that the following operations can be taken when obtaining training samples:

[0129] See also Figure 4 , is a flow chart of a method for constructing training samples provided in an embodiment of the present application, such as Figure 4 As shown, the steps include:

[0130] Step S401, obtaining historical behavior data of each training object in the target application.

[0131] In this step, it is necessary to reiterate that the acquisition, use and processing of training samples must comply with the relevant laws, regulations and standards of relevant countries and regions. It can be implemented within the authorized organization, and the following implementation methods can be implemented using the data of internal members of the organization to identify internal members; or, the relevant data used in the specific implementation are all simulated data.

[0132] In order to improve the personalized churn probability prediction for different objects, each historical behavior data corresponds to only one training object identifier. In other words, the historical behavior data in one historical behavior data only comes from one training object, and different historical behavior data correspond to different training objects.

[0133] Step S402, for each piece of historical behavior data obtained, determine whether the preset data type in each piece of historical behavior data meets the first time condition; if so, execute step S403, otherwise, execute step S404.

[0134] In actual applications, the preset data type can select different data types according to different types of target applications. For example, assuming that the target application is a content access application, the preset data type can be login data, and the judgment condition is whether the historical behavior data has not shown login data for more than a certain period of time; for another example, assuming that the target application is an item transaction application, the preset data type can be transaction time data, and the judgment condition is whether the historical behavior data has not shown transaction time data for more than a certain period of time.

[0135] The specific time data of the first time condition can be determined according to the needs of the user, and this application does not impose any restrictions on this.

[0136] Step S403: Create training samples whose true labels are positive samples based on the corresponding historical behavior data.

[0137] When it is determined that a certain data type in the historical behavior data meets the preset first time condition, the training object corresponding to the historical behavior data can be determined as a lost object, and then the real label corresponding to the historical behavior data is determined as a positive sample.

[0138] In one possible implementation, when creating training samples whose true labels are positive samples based on historical behavior data: first, the churn time of the training object can be determined based on the historical behavior data; then, according to the churn time of the training object and the historical behavior data of the training object, the training feature set of the training object is determined, and the training feature set includes positive samples indicating that the training object is a churn object; finally, the training feature set of the training object and the identifier of the training object are concatenated to obtain the training sample.

[0139] The training feature set includes object portrait features of the training object, behavioral statistical features of the training object in the target application, contextual features of the training object, and real-time behavior sequence features of the object based on time sequence.

[0140] Exemplarily, when determining the training feature set of the training object based on the loss time of the training object and the historical behavior data of the training object: for the object portrait feature and the behavior statistical feature, the features of the training object one day before the loss time are taken; for the object real-time behavior sequence feature, the training object behavior before the loss time is taken as the real-time behavior sequence feature; for the context feature, conversion is performed according to the loss time, such as converting the last login data to today, and subtracting the time difference between the object's loss time and the object's last registration time.

[0141] Step S404: creating training samples whose true labels are negative samples based on the corresponding historical behavior data.

[0142] As described above, based on the historical behavior data that does not satisfy the first time condition, corresponding training samples with true labels as negative samples can be created.

[0143] In one possible implementation, when creating training samples whose true labels are negative samples based on historical behavior data: first, determine a training feature set of the training object based on the current time and the historical behavior data of the training object, where the training feature set includes negative samples indicating that the training object is a non-churn object; then, concatenate the training feature set of the training object and the identifier of the training object to obtain the training sample.

[0144] Since the number of negative samples is much larger than that of positive samples, in order to avoid the model learning too many features of negative samples during training, thereby squeezing out the learning process for positive sample features, after obtaining each training sample and determining its corresponding true label, you can continue to perform the following operations:

[0145] Step S405: Based on the obtained true labels corresponding to the training samples, the training weights corresponding to the training samples are adjusted respectively.

[0146] The training weight is used to indicate the influence of the training sample on the training results during the model training process.

[0147] For example, for positive and negative training samples, the adjustment of training weights can be divided into three possible ways, such as Figure 5A As shown, the training weights corresponding to the positive samples can be increased, or, as Figure 5B As shown, reduce the training weight of negative samples, or as Figure 5C As shown in Figure 2, while increasing the training weight of positive samples, the training weight of negative samples is reduced.

[0148] In this way, the model can improve its learning effect on the features of positive samples during training.

[0149] In addition to adjusting the training weights corresponding to each training sample, another way to reduce the impact of negative samples on the training results is to reduce the number of negative samples collected when collecting historical behavior data.

[0150] Specifically, when executing step S402, when it is determined that the preset data type in the historical behavior data does not meet the first time condition, a corresponding negative sample can be created based on the historical behavior data at intervals. For example, when it is determined that the preset data type in 10 historical behavior data does not meet the first time condition, a training sample with a true label as a negative sample can be created based on the last historical behavior data. In this way, the number of negative samples generated can be reduced from the sampling process, and the impact of negative samples on model training can also be reduced.

[0151] In the present application, in addition to the training feature set, the training samples also include identifiers for determining object preference features. During training, each training object can consider its own preference features to determine the importance of different features and improve the accuracy of model training.

[0152] 3. Model training stage:

[0153] In the embodiment of the present application, when the intent prediction model is trained, the intent prediction model is trained by executing multiple rounds of loop iterations to obtain a trained intent prediction model. Since the operations performed in each round of loop iterations are the same, the method for training the intent prediction model is described by taking one round of loop iterations as an example.

[0154] See also Figure 6 , Figure 6 A flowchart of a method for training an intention prediction model provided in an embodiment of the present application includes the following steps:

[0155] Step S600, determining a training feature set of the training object according to the loss time of the training object and the behavior data of the training object, wherein the training feature set includes a label of the training object.

[0156] Step S601: Concatenate the training feature set of the training object and the identifier of the training object to obtain a training sample.

[0157] It should be noted that the process of constructing the training samples can refer to the content in the above-mentioned training sample construction stage, and will not be repeated here.

[0158] Step S602: input the training sample into the intention prediction model, and output the loss prediction value of the training object.

[0159] In each round of iterative training, all training samples can be used as the input of this round, or some training samples can be extracted as the input of this round. The process of object churn probability prediction can refer to the content in the above model design stage, and will not be repeated here.

[0160] Step S603: training an intention prediction model based on the difference between the churn prediction value of the training object and the label of the training object.

[0161] In an embodiment of the present application, when training the intention prediction model based on the difference value between the churn prediction value of the training object and the label of the training object, the model loss value of the intention prediction model of this iteration is determined based on the difference value between the churn prediction value of the training object and the label of the training object, and the intention prediction model is trained according to the model loss value.

[0162] In one possible implementation, the model loss value can be calculated using any possible loss function, such as a cross entropy loss function (Cross Entropy Loss Function), a mean squared error (MSE) loss function, a squared absolute error loss function, a maximum likelihood loss (Likelihood Loss, LHL) function, etc. Of course, other possible loss functions may also be used, and the embodiments of the present application are not limited to this.

[0163] In one possible implementation, when training the intent prediction model according to the model loss value, it is determined whether the intent prediction model reaches the convergence condition. If the convergence condition is not reached, the model parameters are adjusted based on the model loss value. If the convergence condition is reached, the trained intent prediction model is output.

[0164] The convergence conditions may include one or more of the following conditions:

[0165] (1) The number of iterations reaches the set threshold.

[0166] (2) The model loss value is less than the set loss threshold.

[0167] For example, when the number of iterations does not exceed the preset number threshold, and the model loss value is not less than the set loss threshold, it is considered that the current prediction model to be trained does not meet the convergence condition, and the model parameters need to be adjusted to continue training. After the parameter adjustment, the next iterative training process is entered, that is, jumping to step S602.

[0168] In a possible implementation, when the model still does not meet the convergence condition, the model weight parameters can be updated by the gradient descent method, and the training can be continued with the updated model weight parameters.

[0169] When the number of iterations exceeds the preset threshold, or the model loss value is less than the set loss threshold, it is considered that the current prediction model to be trained has met the convergence conditions, the model training is completed, and the intent prediction model is obtained.

[0170] In the embodiment of the present application, in order to verify the effect of the trained intention prediction model, the intention prediction model can also be tested using training samples. If the evaluation index reaches the preset evaluation effect, the intention prediction model can be used in the actual object loss probability prediction process. If the model evaluation fails, it is necessary to continue training and testing until the prediction model to be trained reaches the preset evaluation effect.

[0171] See also Figure 7 , Figure 7 A schematic diagram of a specific implementation method of intention prediction model training provided in an embodiment of the present application, from Figure 7 It can be seen that:

[0172] First, the object portrait data of the training object, the behavioral statistics data of the training object in the target application, the real-time behavioral data of the training object in the target application, and the context data of the training object are converted and processed respectively through the embedding layer to obtain the corresponding behavioral semantic vectors, and then the obtained behavioral semantic vectors are spliced ​​to obtain the historical behavioral semantic vectors of the behavioral data; similarly, the historical preference data determined based on the identification of the training object is converted and processed through the embedding layer to obtain the corresponding historical preference semantic vectors;

[0173] Next, the historical behavior semantic vector and the historical preference semantic vector are input into the first feature processing layer to obtain the historical initial feature vector output by the first feature processing layer; and the historical initial feature vector and the historical preference semantic vector are input into the second feature processing layer to obtain the historical target feature vector output by the first feature processing layer;

[0174] Then, the historical target feature vector is reduced in dimension through the fully connected layer to obtain the churn prediction value of the training object;

[0175] Finally, the churn prediction value and label are input into the loss function layer to determine the loss value, and whether the model convergence condition is met is judged based on the loss value. If the model convergence condition is met, the trained intent prediction model is output, otherwise the parameter value of the intent prediction model is adjusted.

[0176] In the present application, when training the intention prediction model based on training samples, the preference data of the object identifier that best represents the essential characteristics of the object is added for training, which can explicitly learn the importance of different features to different objects, enhance the personalized fitting ability of the model, and thus improve the accuracy of the model's intention probability prediction.

[0177] Example 2: Model application process

[0178] In a possible implementation, after the model training is completed, the intention prediction model can be used to participate in the actual object loss probability prediction process, that is, the intention prediction model is used to predict the intention probability of the target object. Specifically, the behavior data and identification of the target object are first obtained, and then the behavior data of the target object and the preference data determined based on the identification of the target object are input into the intention prediction model, and the intention prediction is performed through the intention prediction model, and the intention probability of the target object is output. The intention probability is used to represent the probability of loss of the target object.

[0179] See also Figure 8 , is a flow chart of a method for intention prediction provided in an embodiment of the present application, comprising the following steps:

[0180] Step S800, obtaining a target feature vector of the target object based on the behavior feature vector and the preference feature vector of the target object.

[0181] In the specific implementation, the target feature vector of the target object is obtained based on the behavior feature vector and preference feature vector of the target object through the intention prediction model; the specific structure of the intention prediction model can be found in FIG. 3A to FIG. 3C The structure shown.

[0182] In the embodiment of the present application, when the target feature vector of the target object is obtained based on the behavior feature vector and the preference feature vector of the target object through the intention prediction model:

[0183] First, the behavior data and preference data input into the intent prediction model are obtained; illustratively, the corresponding behavior data of the target object is obtained based on the first time range from the current moment, and the preference data of the target object is obtained based on the identifier of the target object. The identifier of the target object can be associated with all the behavior data generated by the target object since the target application is used.

[0184] Among them, the behavior data includes but is not limited to: object portrait data of the target object, behavior statistics data of the target object in the target application, real-time behavior data of the target object in the target application, and context data of the target object.

[0185] It should be noted that for the first time range, Fig. 9As shown, the corresponding time range is a period of time from the current moment. Therefore, the recent period can be used to summarize the time period covered by the first time range, and the specific length of time can be determined based on actual usage requirements. For example, the first time range can be set to 7 natural days, or 30 natural days, etc., and this application does not impose any restrictions on this.

[0186] Then, based on the intention prediction model, feature extraction is performed on the behavior data to obtain a behavior semantic feature vector, and feature extraction is performed on the preference data to obtain a preference semantic vector; for example, see Fig.10 , Fig.10 A schematic diagram of obtaining the probability of intention through an intention prediction model provided in an embodiment of the present application, from Fig.10 It can be seen that the object portrait data, behavior statistics data, real-time behavior data, and target context data are input into the embedding layer of the intention prediction model, and the features of each behavior data are extracted through the embedding layer to obtain the corresponding semantic feature vector, and the semantic feature vectors of all behavior data are spliced ​​to obtain the behavior semantic features; and the preference data obtained based on the identification of the target object is input into the embedding layer of the intention prediction model, and the feature extraction of the preference data is performed through the embedding layer to obtain the corresponding preference semantic vector.

[0187] Finally, the behavior feature vector is determined based on the behavior semantic vector, and the preference feature vector is determined based on the preference semantic vector, and the target feature vector of the target object is obtained based on the behavior feature vector and the preference feature vector of the target object; for example, see Fig.10 ,from Fig.10 It can be seen that the behavior semantic vector and the preference semantic vector are input into at least one feature processing layer of the intention prediction model, and at least one feature processing layer is used to determine the behavior feature vector based on the behavior semantic vector, determine the preference feature vector based on the preference semantic vector, and obtain the target feature vector of the target object based on the behavior feature vector and the preference feature vector of the target object. For a detailed description, see Figure 3A-3C The relevant content will not be repeated here.

[0188] Step S801 , predicting the intention of the target object based on the target feature vector to obtain the intention probability of the target object, where the intention probability is used to represent the probability of loss of the target object.

[0189] Continue to see Fig.10 ,from Fig.10 It can be seen that through the fully connected layer of the intention prediction model, the intention of the target object is predicted based on the target feature vector to obtain the intention probability of the target object.

[0190] In the present application, when predicting the probability of target object loss, the target feature vector of the target object is first obtained based on the behavioral feature vector and preference feature vector of the target object; it can be seen that when determining the target feature vector, not only the behavioral feature vector of the object is taken into account, but also the preference feature vector of the object is added. The preference feature vector can better represent the degree of behavioral preference of the object, that is, it reflects the personalized characteristics (or essential characteristics) of the object, so the obtained target feature vector can better represent the importance of different behavioral characteristics to the target object; further, based on the target feature vector, the intention of the target object is predicted, and the intention probability used to represent the probability of target object flow is obtained. The importance of different behavioral characteristics to the target object can be learned, the personalized fitting of the target object is enhanced, and the accuracy of predicting the probability of loss is improved.

[0191] After completing the prediction of the object loss probability, if the target object is determined to be an object with a higher loss probability based on the obtained intention probability of the target object, or the intention probability exceeds the probability threshold, the object executes the corresponding recovery strategy based on the preference feature vector obtained during the prediction process.

[0192] In a possible implementation, when it is determined that the target object's probability of churn for the target application exceeds a preset probability threshold, based on the target object's preference feature vector for the target application, the target object's preferred content in the target application is obtained; then, based on the preferred content, recovery information is generated and sent to the target object, thereby improving the success rate of retention. Each element in the preference feature vector is associated with a content and represents the target object's operation status or preference for the content.

[0193] For example, if a certain object is more sensitive to the price of items, when generating the object's recovery information, coupons for different items can be generated and sent to the object's account to retain the object.

[0194] When generating recovery information for different objects, it can be done in the following different ways:

[0195] In a possible implementation, based on the preferred content corresponding to the target object, the recommendation weight of each content in the target application is adjusted, and then based on the adjusted recommendation weight, corresponding redemption information is generated and sent to the target object.

[0196] Exemplarily, assuming that based on the preferred content, the target object's preference value for different types of content is determined, then the recommendation weight of the content to be recommended in the target application can be adjusted based on the preference value, so that the recommendation weight is positively correlated with the preference value corresponding to the target object. In this way, more preferred content can be recommended to the target object, thereby improving the success rate of retaining the target object.

[0197] In another possible implementation, based on the preference content corresponding to the target object and a preset recovery strategy, recovery information for the target object is generated, and the recovery information is sent to the target object.

[0198] The target application can pre-configure corresponding recovery strategies for different types of objects, so that corresponding recovery information can be generated based on the characteristics and preferences of the target objects. For example, for price-sensitive objects, different coupons can be generated to recover the target objects, while for socially active objects, different friend information can be generated and recovery information containing these friend information can be sent to the target objects, and so on.

[0199] The above two methods can be applied separately or in combination during implementation. The separate application method is as above and will not be elaborated here. The combined application method is as follows: set multiple probability thresholds. When the loss probability exceeds the first probability threshold but does not exceed the second probability threshold, the first implementation method can be used to send recovery information to the object without perception, so as to avoid the object from feeling disturbed and improve the object's usage experience; when the loss probability exceeds the second probability threshold, it means that the loss probability of the object is high, and the second implementation method can be used to send additional recovery information to the object to improve the corresponding recovery success rate.

[0200] Based on the same inventive concept, the present application embodiment also provides an apparatus 1100 for predicting intention, see Fig.11 , the intention prediction device 1100 includes:

[0201] An obtaining unit 1101 is used to obtain a target feature vector of the target object based on the behavior feature vector and the preference feature vector of the target object;

[0202] The prediction unit 1102 is used to predict the intention of the target object based on the target feature vector to obtain the intention probability of the target object, and the intention probability is used to represent the probability of loss of the target object.

[0203] In a possible implementation, the obtaining unit 1101 is specifically configured to:

[0204] Through the intention prediction model, the target feature vector of the target object is obtained based on the behavior feature vector and preference feature vector of the target object.

[0205] In a possible implementation, the intent prediction model includes a feature extraction network, a gate network, and a fusion network. Through the intent prediction model, the obtaining unit 1101 is specifically used for:

[0206] Through the feature extraction network, feature extraction is performed on the behavior semantic vector associated with the behavior data of the target object to obtain the behavior feature vector of the target object;

[0207] Through the gate network, feature extraction is performed on the preference semantic vector associated with the target object's identification to obtain the preference feature vector of the target object. The dimension of the preference feature vector is the same as that of the behavior feature vector.

[0208] Through the fusion network, based on the first element in the preference feature vector, the second element in the behavior feature vector with the same position information as the first element is fused to obtain the target feature vector.

[0209] In a possible implementation, the gate network includes: a first fully connected layer and a second fully connected layer; the obtaining unit 1101 is specifically used for:

[0210] Through the first fully connected layer, the preference semantic vector associated with the target object's identification is transformed into a nonlinear feature vector to obtain the initial preference vector;

[0211] Through the second fully connected layer, the dimension of the initial preference vector is transformed to obtain the preference feature vector.

[0212] In a possible implementation, the obtaining unit 1101 is specifically configured to:

[0213] The first element in the preference feature vector is used as the preference weight of the second element in the behavior feature vector that has the same position information as the first element;

[0214] weighting the corresponding second element in the behavior feature vector based on the preference weight to obtain a weighted behavior feature vector;

[0215] The weighted behavior feature vector is used as the target feature vector.

[0216] In a possible implementation, the intention prediction model further includes a fully connected layer, and the prediction unit 1102 is specifically used to:

[0217] Through the fully connected layer, the behavior of the target object is predicted based on the target feature vector to obtain the intention probability of the target object.

[0218] Based on the same inventive concept, the present application embodiment also provides a device 1200 for training an intention prediction model, see Fig.12 , the apparatus 1200 for training the intention prediction model includes:

[0219] A determination unit 1201 is used to determine a training feature set of the training object according to the loss time of the training object and the behavior data of the training object, where the training feature set includes a label of the training object;

[0220] A concatenation unit 1202 is used to concatenate a training feature set of a training object and an identifier of the training object to obtain a training sample;

[0221] An input unit 1203 is used to input the training sample into the intention prediction model and output the loss prediction value of the training object;

[0222] The training unit 1204 is used to train the intention prediction model based on the difference between the churn prediction value of the training object and the label of the training object.

[0223] In a possible implementation, the intention prediction model includes a feature extraction network, a gate network, a fusion network, and a fully connected layer; the input unit 1203 is specifically used for:

[0224] Through the feature extraction network, feature extraction is performed on the behavior semantic vector associated with the behavior data of the training object to obtain the historical behavior feature vector;

[0225] Through the gate network, feature extraction is performed on the preference semantic vector associated with the identification of the training object to obtain the historical preference feature vector of the training object. The dimension of the historical preference feature vector is the same as that of the historical behavior feature vector.

[0226] Through the fusion network, based on the first element in the historical preference feature vector, the second element in the historical behavior feature vector with the same position information as the first element is fused to obtain the historical target feature vector;

[0227] Through the fully connected layer, the historical target feature vector is reduced in dimension to obtain the churn prediction value of the training object.

[0228] In a possible implementation, the gate network includes: a first fully connected layer and a second fully connected layer, and the input unit 1203 is specifically used to:

[0229] Through the first fully connected layer, the preference semantic vector associated with the identification of the training object is transformed into a nonlinear feature vector to obtain the initial vector of historical preference;

[0230] Through the second fully connected layer, the dimension of the historical preference initial vector is transformed to obtain the historical preference feature vector.

[0231] It should be noted that, although several units (or modules) of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the implementation of the present application, the features and functions of two or more units (or modules) described above can be concretized in one unit (or module). Conversely, the features and functions of one unit (or module) described above can be further divided into multiple units (or modules) to be concretized. Of course, when implementing the present application, the functions of each unit (or module) can also be implemented in the same one or more software or hardware.

[0232] After introducing the method and device for intention prediction and intention prediction model training of an exemplary implementation of the present application, another exemplary implementation of the present application, an electronic device, is introduced next.

[0233] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0234] The electronic device provided in the embodiment of the present application may include at least a processor and a memory. The memory stores a program code, and when the program code is executed by the processor, the processor executes any step of the method of intent prediction and intent prediction model training in various exemplary implementations of the present application.

[0235] In a possible implementation, the electronic device may be a server, such as Figure 1 In this embodiment, the structure of the electronic device is as follows: Fig.13 As shown, it may include at least a memory 1301 , a communication module 1303 , and at least one processor 1302 .

[0236] The memory 1301 is used to store computer programs executed by the processor 1302. The memory 1301 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0237] The memory 1301 may be a volatile memory, such as a random-access memory (RAM); the memory 1301 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 1301 may be any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1301 may be a combination of the above memories.

[0238] The processor 1302 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 1302 is used to implement the above-mentioned method of intent prediction and intent prediction model training when calling the computer program stored in the memory 1301.

[0239] The communication module 1303 is used to communicate with terminal devices and other servers.

[0240] The specific connection medium between the memory 1301, the communication module 1303 and the processor 1302 is not limited in the embodiment of the present application. Fig.13 In the embodiment, the memory 1301 and the processor 1302 are connected via a bus 1304. The bus 1304 is connected to the processor 1302 via a bus 1304. Fig.13 The connections between the other components are only for illustration and are not intended to be limiting. The bus 1304 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Fig.13 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or only one type of bus.

[0241] The memory 1301 stores a computer storage medium, and the computer storage medium stores computer executable instructions, which are used to implement the method of intention prediction and intention prediction model training in the embodiment of the present application. The processor 1302 is used to execute the above-mentioned method of intention prediction and intention prediction model training.

[0242] In another possible implementation, the electronic device may also be other electronic devices, such as Figure 1 The terminal device 101 is shown in FIG. In this embodiment, the structure of the electronic device can be as follows: Fig.14 As shown, it includes: a communication component 1410, a memory 1414, a display unit 1430, a camera 1440, a sensor 1450, an audio circuit 1460, a Bluetooth module 1470, a processor 1480 and other components.

[0243] The communication component 1410 is used to communicate with the server. In some embodiments, a wireless fidelity (WiFi) module may be included. The WiFi module belongs to a short-range wireless transmission technology. The electronic device can help the object to send and receive information through the WiFi module.

[0244] The memory 1414 can be used to store software programs and data. The processor 1480 executes various functions and intent prediction of the terminal device 101 by running the software programs or data stored in the memory 1414. The memory 1414 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. The memory 1414 stores an operating system that enables the terminal device 101 to run. In the present application, the memory 1414 can store an operating system and various application programs, and can also store a computer program for executing the method of intent prediction and intent prediction model training provided in the embodiment of the present application.

[0245] The display unit 1430 can also be used to display information input by the object or information provided to the object and a graphical user interface (GUI) of various menus of the terminal device 101. Specifically, the display unit 1430 may include a display screen 1432 disposed on the front of the terminal device 101. The display screen 1432 may be configured in the form of a liquid crystal display, a light-emitting diode, etc. The display unit 1430 can be used to display the recovery information display interface, the model training interface, etc. in the embodiment of the present application.

[0246] The display unit 1430 can also be used to receive input digital or character information, and generate signal input related to the object setting and function control of the terminal device 101. Specifically, the display unit 1430 may include a touch screen 1431 set on the front of the terminal device 101, which can collect touch operations of objects on or near it, such as clicking a button, dragging a scroll box, etc.

[0247] The touch screen 1431 can be covered on the display screen 1432, or the touch screen 1431 and the display screen 1432 can be integrated to realize the input and output functions of the physical terminal device 101. The integrated touch screen can be referred to as a touch display screen. In this application, the display unit 1430 can display the application and the corresponding operation steps.

[0248] The camera 1440 can be used to capture static images, and the subject can publish the images taken by the camera 1440 through the application. The camera 1440 can be one or more. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the processor 1480 to convert it into a digital image signal.

[0249] The physical terminal device may also include at least one sensor 1450, such as an acceleration sensor 1451, a distance sensor 1452, a fingerprint sensor 1453, and a temperature sensor 1454. The terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, and a motion sensor.

[0250] The audio circuit 1460, the speaker 1461, and the microphone 1462 can provide an audio interface between the object and the terminal device 101. The audio circuit 1460 can transmit the electrical signal converted from the received audio data to the speaker 1461, which is converted into a sound signal for output. The physical terminal device 101 can also be configured with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 1462 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1460 and converted into audio data, and then the audio data is output to the communication component 1410 to be sent to, for example, another physical terminal device 101, or the audio data is output to the memory 1414 for further processing.

[0251] The Bluetooth module 1470 is used to exchange information with other Bluetooth devices having Bluetooth modules through the Bluetooth protocol. For example, the physical terminal device can establish a Bluetooth connection with a wearable electronic device (such as a smart watch, smart glasses) that also has a Bluetooth module through the Bluetooth module 1470 to exchange data.

[0252] The processor 1480 is the control center of the physical terminal device. It uses various interfaces and lines to connect various parts of the entire terminal. It executes various functions of the terminal device and processes data by running or executing software programs stored in the memory 1414 and calling data stored in the memory 1414. In some embodiments, the processor 1480 may include one or more processing units; the processor 1480 may also integrate an application processor and a baseband processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the baseband processor mainly processes wireless communications. It is understandable that the above-mentioned baseband processor may not be integrated into the processor 1480. In the present application, the processor 1480 can run the operating system, application programs, user interface display and touch response, as well as the method of intention prediction and intention prediction model training in the embodiment of the present application. In addition, the processor 1480 is coupled to the display unit 1430.

[0253] In some possible embodiments, various aspects of the method for intent prediction and intent prediction model training provided in the present application can also be implemented in the form of a program product, which includes a computer program. When the program product is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of the method for intent prediction according to various exemplary embodiments of the present application described above in this specification, or execute the steps of the method for intent prediction model training according to various exemplary embodiments of the present application described above in this specification.

[0254] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0255] The program product of the embodiment of the present application may adopt a portable compact disk read-only memory (CD-ROM) and include a computer program, and can be run on an electronic device. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which can be used by or in combination with a command execution system, apparatus, or device.

[0256] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a command execution system, apparatus, or device.

[0257] The computer program embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0258] Computer programs for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The computer program may be executed entirely on the electronic device, partially on the electronic device, as a separate software package, partially on the electronic device and partially on a remote electronic device, or entirely on the remote electronic device. In cases involving remote electronic devices, the remote electronic device may be connected to the electronic device via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external electronic device (e.g., via the Internet using an Internet service provider).

[0259] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0260] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0261] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0262] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0263] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0264] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0265] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for intention prediction, characterized in that: The method comprises: Based on the behavior feature vector and the preference feature vector of the target object, obtaining a target feature vector of the target object; The intention of the target object is predicted based on the target feature vector to obtain the intention probability of the target object, and the intention probability is used to represent the probability of loss of the target object.

2. The method according to claim 1, characterized in that The step of obtaining a target feature vector of the target object based on the behavior feature vector and the preference feature vector of the target object includes: Through the intention prediction model, based on the behavior feature vector and the preference feature vector of the target object, a target feature vector of the target object is obtained.

3. The method according to claim 2, characterized in that The intention prediction model includes a feature extraction network, a gate network, and a fusion network. The intention prediction model is used to obtain a target feature vector of the target object based on the behavior feature vector and the preference feature vector of the target object, including: Through the feature extraction network, feature extraction is performed on the behavior semantic vector associated with the behavior data of the target object to obtain the behavior feature vector of the target object; By means of the gate network, feature extraction is performed on the preference semantic vector associated with the identification of the target object to obtain a preference feature vector of the target object, wherein the dimension of the preference feature vector is the same as the dimension of the behavior feature vector; Through the fusion network, based on the first element in the preference feature vector, the second element in the behavior feature vector with the same position information as the first element is fused to obtain the target feature vector.

4. The method according to claim 3, characterized in that The gate network includes: a first fully connected layer and a second fully connected layer; The step of extracting features of the preference semantic vector associated with the identifier of the target object through the gate network to obtain the preference feature vector of the target object includes: Through the first fully connected layer, nonlinear feature transformation is performed on the preference semantic features associated with the identifier of the target object to obtain a preference initial vector; The dimension of the preference initial vector is transformed through the second fully connected layer to obtain the preference feature vector.

5. The method according to claim 3, characterized in that The step of fusing the second element in the behavior feature vector having the same position information as the first element based on the first element in the preference feature vector to obtain the target feature vector includes: Using the first element in the preference feature vector as the preference weight of the second element in the behavior feature vector having the same position information as the first element; Performing weighted processing on the corresponding second element in the behavior feature vector based on the preference weight to obtain a weighted behavior feature vector; The weighted behavior feature vector is used as the target feature vector.

6. The method according to claim 2, characterized in that The intention prediction model further includes a fully connected layer, and the behavior of the target object is predicted based on the target feature vector to obtain the intention probability of the target object, including: Through the fully connected layer, the behavior of the target object is predicted based on the target feature vector to obtain the intention probability of the target object.

7. A method for training an intention prediction model, characterized in that: The method comprises: Determine a training feature set of the training object according to the loss time of the training object and the behavior data of the training object, wherein the training feature set includes a label of the training object; Concatenating the training feature set of the training object and the identifier of the training object to obtain a training sample; Inputting the training sample into the intention prediction model, and outputting the loss prediction value of the training object; The intention prediction model is trained based on the difference value between the churn prediction value of the training object and the label of the training object.

8. The method according to claim 7, characterized in that The intention prediction model includes a feature extraction network, a gate network, a fusion network, and a fully connected layer; the step of inputting the training sample into the intention prediction model and outputting the loss prediction value of the training object includes: Through the feature extraction network, feature extraction is performed on the behavior semantic vector associated with the behavior data of the training object to obtain a historical behavior feature vector; Through the gate network, feature extraction is performed on the preference semantic vector associated with the identification of the training object to obtain a historical preference feature vector of the training object, where the dimension of the historical preference feature vector is the same as the dimension of the historical behavior feature vector; By means of the fusion network, based on the first element in the historical preference feature vector, a second element in the historical behavior feature vector having the same position information as the first element is fused to obtain a historical target feature vector; The historical target feature vector is reduced in dimension through the fully connected layer to obtain the churn prediction value of the training object.

9. The method according to claim 8, characterized in that The gate network includes: a first fully connected layer and a second fully connected layer. The gate network is used to extract features of the preference semantic vector associated with the identifier of the training object to obtain the historical preference feature vector of the training object, including: Through the first fully connected layer, nonlinear feature transformation is performed on the behavior semantic vector associated with the identification of the training object to obtain an initial vector of historical preference; The dimension of the historical preference initial vector is transformed through the second fully connected layer to obtain the historical preference feature vector.

10. A device for predicting intention, characterized in that: The device comprises: an obtaining unit, configured to obtain a target feature vector of the target object based on the behavior feature vector and the preference feature vector of the target object; A prediction unit is used to predict the intention of the target object based on the target feature vector to obtain the intention probability of the target object, and the intention probability is used to represent the probability of loss of the target object.

11. A device for training an intention prediction model, characterized in that: The device comprises: A determination unit, configured to determine a training feature set of the training object according to the loss time of the training object and the behavior data of the training object, wherein the training feature set includes a label of the training object; A concatenation unit, configured to concatenate the training feature set of the training object and the identifier of the training object to obtain a training sample; An input unit, used to input the training sample into the intention prediction model and output the loss prediction value of the training object; A training unit is used to train the intention prediction model based on the difference value between the churn prediction value of the training object and the label of the training object.

12. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 9.

13. A computer-readable storage medium, characterized in that: It includes program codes. When the program codes are run on an electronic device, the program codes are used to enable the electronic device to execute the steps of the method according to any one of claims 1 to 9.

14. A computer program product, characterized in that The invention comprises a computer program, which implements the steps of the method according to any one of claims 1 to 9 when being executed by a processor.

Images